Efficient large-model full-process encryption method

Abstract

The invention belongs to the technical field of large model security, and particularly relates to an efficient large model full-process encryption method. Comprising the steps of traversing a large model and performing operation deployment; the GPU calculates an attention score matrix and positions important parameters in a Q matrix and a K matrix; partitioning the Q matrix and the K matrix; the GPU extracts important parameters from each block and constructs a block parameter packet; after the TEE verifies the block parameter packet, OTP encryption is carried out on the important parameters, and encrypted blocks are generated; the GPU restores the encryption parameter to a parameter position corresponding to the original block to form a semi-encryption block; executing parallel linear ciphertext operation on the semi-encrypted blocks to generate ciphertext operation result blocks; the TEE performs block decryption on the ciphertext operation result, then performs nonlinear operation, and performs secondary OTP encryption on the nonlinear operation result to generate a secondary encryption result; and the GPU directly uses the secondary encryption result to carry out linear calculation, and then transmits the result back to the TEE to decrypt until a reasoning result is obtained, so that the security of sensitive parameter encryption is remarkably improved, and the efficiency is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of large-scale model security technology, specifically relating to an efficient full-process encryption method for large-scale models. More specifically, it is an optimized method for the entire process of "precise encryption of block parameters - ciphertext operation - secure restoration". Background Technology

[0002] As the scale of large model parameters exceeds hundreds of billions, the inference process relies on the parallel computing capabilities of GPUs to achieve efficient linear computation. At the same time, it is necessary to ensure the security of sensitive parameters through TEE. However, existing technologies have four core pain points and cannot meet the three-in-one requirements of "efficiency-security-accuracy": Existing solutions mostly divide the data into blocks according to the "GPU register width" (such as multiples of 64-bit / 128-bit), without combining the "distribution of important parameters" of the large model attention mechanism, resulting in: ① poor compatibility between the blocks and GPU tensor cores (such as NVIDIA 16×16), requiring data to be split during computation, increasing latency by 30%-40%; ② important parameters are scattered in multiple blocks, resulting in redundant encrypted transmission (full matrix transmission, with non-important parameters accounting for more than 80%), and PCIe channel load exceeding 60%.

[0003] While some solutions attempt to encrypt critical parameters, they lack a mechanism linking "unique block identifier - parameter coordinates": ① When encrypted parameters are sent back to the GPU, the original block position cannot be accurately located, leading to parameter misalignment (e.g., filling Q matrix block parameters into K matrix), reducing inference accuracy to below 85%; ② The number of restored parameters is not verified, and missing or duplicate encrypted parameters cannot be detected in time, further exacerbating result deviations. Existing encryption schemes have two types of risks: ① The OTP key length does not match the parameter data volume, resulting in some parameters being unencrypted or redundantly encrypted, leading to incomplete security protection. In summary, there is an urgent need for a full-process method that can achieve "block adaptation to GPU hardware - accurate encryption of critical parameters - secure key isolation" to solve the efficiency, security, and compatibility bottlenecks of existing technologies. Summary of the Invention

[0004] To address the problems of "poor block adaptability," "encryption and restoration misalignment," "high key risk," and "weak hardware compatibility" in existing large-scale model encryption methods, this invention proposes an efficient full-process encryption method for large-scale models, which includes:

[0005] S1: Traverse the operation types of each layer of the large model and deploy the linear operation part on the GPU and the non-linear operation part on the TEE;

[0006] S2: The large model receives user input data, and the GPU calculates the Q matrix and K matrix based on the user input data. The attention score matrix is ​​generated by multiplying the Q matrix and K matrix. High contribution elements in the attention score matrix are selected, and important parameters in the Q matrix and K matrix are located in reverse based on the high contribution elements.

[0007] S3: Divide the Q matrix and K matrix into blocks and assign a unique identifier to each block;

[0008] S4: The GPU extracts important parameters from each block and constructs a block parameter packet with a unique block identifier and check value, which is then passed to the TEE through the PCIe secure channel; non-important parameters are stored in the GPU plaintext cache.

[0009] S5: The TEE verifies the checksum of the block parameter packet. After the verification is successful, it performs encryption processing on the important parameters in the block to generate an encrypted block; the encrypted block is then sent back to the GPU through the PCIe secure channel.

[0010] S6: The GPU receives the encrypted block and restores the encrypted parameters in the encrypted block to the corresponding parameter positions of the original block to form a semi-encrypted block; the GPU computing unit is called to perform parallel linear ciphertext operation on the semi-encrypted block to generate a ciphertext operation result block with a unique block identifier;

[0011] S7: The GPU sends the ciphertext operation results in blocks to the TEE. The TEE calls the corresponding encryption key to decrypt the ciphertext operation results in blocks. It then performs non-linear operations on the decrypted results and uses a preset encryption algorithm to encrypt the non-linear operation results again, generating a second encrypted result that is sent back to the GPU.

[0012] S8: The GPU performs linear operations on the secondary encryption result, sends the result back to the TEE for decryption, and continues until the final inference result is obtained.

[0013] Preferably, in step S2, the process of filtering high-contribution elements in the attention score matrix and locating important parameters in the Q and K matrices based on these high-contribution elements includes:

[0014] Calculate the mean μ and standard deviation σ of the elements in the attention score matrix, set a threshold T = μ + 1.2 × σ, and filter out high contribution elements with values ​​≥ T in the score matrix; based on the coordinates (row i, column j) of the high contribution element in the score matrix, reverse locate the parameter corresponding to it in the i-th row of the Q matrix and the j-th column of the K matrix, and use this parameter as the important parameter.

[0015] Preferably, in step S3, when dividing the Q matrix and K matrix into blocks, the block size is an integer multiple of the GPU tensor core computation granularity.

[0016] Preferably, in step S5, the process of encrypting important parameters in the block includes:

[0017] TEE uses the OTP encryption algorithm and generates the OTP key through a hardware random number generator. The length of the OTP key is exactly the same as the amount of data of the important parameters in the block.

[0018] TEE performs OTP XOR encryption on important parameters in the block to generate encrypted blocks;

[0019] The TEE stores the mapping relationship between "block unique identifier - OTP key" in the TEE's internal secure storage area for subsequent decryption calls.

[0020] Preferably, in step S6, the process of restoring the encrypted parameters to the parameter positions corresponding to the original block includes:

[0021] The GPU associates the original block corresponding to the encrypted block with non-critical parameters in the plaintext buffer based on the unique identifier of the encrypted block.

[0022] Based on the important parameter coordinates recorded in the block parameter package, the encryption parameters are accurately backfilled into the corresponding coordinate positions of the original block to form a semi-encrypted block;

[0023] The GPU counts the number of encrypted parameters after backfilling and compares it with the total number of important parameters for the block selected in step S2. If the counts are inconsistent, an encrypted block retransmission request is sent to the TEE to ensure the integrity of the parameter restoration.

[0024] Preferably, in step S7, the preset encryption algorithm is the OTP encryption algorithm.

[0025] The beneficial effects of this invention are as follows: In terms of inference efficiency, for large models with hundreds of billions of parameters, the inference latency of this invention can be controlled within the range of 350-450ms, while the inference latency of existing technologies is generally 600-800ms. In terms of data transmission optimization, through the collaborative optimization of block adaptation of GPU tensor cores and local encryption of important parameters, this solution only transmits important parameters in the blocks, accounting for 10%-20% of the total matrix data volume, while existing technologies often need to transmit the entire matrix data (transmission volume is 100%). By accurately selecting important parameters and reducing redundant data transmission, this solution reduces the data transmission volume by 80%-90%, significantly reducing the PCIe channel load and avoiding the impact of transmission bottlenecks on overall inference efficiency. Regarding TEE resource usage control, in this solution, the TEE only needs to perform encryption operations on important parameters within the block, with the computational load accounting for 10%-20% of the total matrix encryption load. Existing technologies, due to the need to encrypt the entire matrix, often have the TEE computational load at 100% saturation. This solution reduces the TEE computational load by 80%-90% by narrowing the TEE's computational scope, preventing the TEE from becoming a performance bottleneck in the inference process. In terms of key security protection, the key leakage risk of this invention is <0.1%, as the key is generated, stored, and used entirely within the TEE, without being transmitted to external storage areas. In contrast, existing technologies, due to key transmission or multi-block key reuse, have a key leakage risk >25%. This solution, through key isolation and independent generation mechanisms, reduces the key leakage risk by 99.6%, significantly improving the security of sensitive parameter encryption. Attached Figure Description

[0026] Figure 1 This is a flowchart of the efficient large-scale model full-process encryption method in this invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] This invention proposes an efficient full-process encryption method for large models, such as... Figure 1 As shown, the method includes the following:

[0029] S1: Traverse the operation types of each layer of the large model and deploy the linear operation part on the GPU and the non-linear operation part on the TEE.

[0030] First, the model is segmented and deployed. Specifically, the operation types of each layer of the large model are traversed, and linear operations (such as matrix multiplication and FFN layers) are deployed on the GPU, while nonlinear operations (such as activation functions and attention modules) are deployed on the TEE. Linear operations rely on the parallel computing capabilities of the GPU to improve efficiency, while nonlinear operations are protected by the hardware isolation features of the TEE.

[0031] S2: The large model receives user input data, and the GPU calculates the Q matrix and K matrix based on the user input data. The attention score matrix is ​​generated by multiplying the Q matrix and K matrix. High contribution elements in the attention score matrix are selected, and important parameters in the Q matrix and K matrix are located in reverse based on the high contribution elements.

[0032] The GPU calculates the attention score matrix in the attention module based on the input samples of the original large model (such as the token sequence in a typical text reasoning task). Specifically, it calculates the Q matrix and K matrix based on the user input data, and multiplies the Q matrix and K matrix to obtain the attention score matrix. The element values ​​of this matrix reflect the correlation strength between tokens, providing a basis for locating key parameters.

[0033] Filter high-contribution elements from the attention score matrix and use these high-contribution elements to deduce important parameters in the Q and K matrices. Specifically:

[0034] Calculate the mean μ and standard deviation σ of the elements in the attention score matrix, set a threshold T = μ + 1.2 × σ, and filter out high contribution elements with values ​​≥ T in the score matrix; based on the coordinates (row i, column j) of the high contribution element in the score matrix, reverse locate the parameter corresponding to it in the i-th row of the Q matrix and the j-th column of the K matrix, and this parameter is the important parameter.

[0035] S3: Divide the Q matrix and K matrix into blocks and assign a unique identifier to each block.

[0036] The block size is set to an integer multiple of the GPU tensor core computation granularity (including 16×16 for NVIDIA GPUs and 32×32 for AMD GPUs) to ensure that the blocks are adapted to the GPU parallel computing units.

[0037] Unique identifier for each block: It adopts the format of "matrix type-block number-layer number" (e.g., the 1st block and the 3rd layer of the Q matrix are recorded as "Q-01-03"). The identifier is 16 bytes long and is used for the association matching of block parameter encryption, transmission, and restoration.

[0038] S4: The GPU extracts important parameters from each block and constructs a block parameter package with a unique block identifier and check value, which is then passed to the TEE through the PCIe secure channel; non-important parameters are stored in the GPU plaintext cache.

[0039] The GPU extracts important parameters from each block. The data format of the block parameter packet is: block unique identifier (16 bytes) + important parameter data in the block (n bytes, where n is the total number of bytes of important parameters, and each important parameter is stored as 4 bytes in float32 format) + SHA-256 check value (32 bytes).

[0040] When the GPU transmits block parameter packets through the PCIe secure channel, the key for the channel SM4 algorithm is generated by a hardware random number generator within the TEE and is automatically updated every 24 hours to prevent channel data from being intercepted and parsed.

[0041] Non-critical parameters are not entered into the TEE and are stored in the GPU's plaintext cache (protected by the GPU's security registers, allowing only reading and prohibiting modification).

[0042] S5: The TEE verifies the checksum of the block parameter packet. After the verification is successful, it performs encryption processing on the important parameters in the block to generate an encrypted block. The encrypted block is then sent back to the GPU through the PCIe secure channel.

[0043] TEE verifies the checksum of the block parameter packet. If the verification passes, it performs encryption processing on the important parameters in the block. Specifically:

[0044] TEE uses the OTP (One-Time Password) encryption algorithm. It generates the OTP key through a hardware random number generator (including Intel SGX's RDRAND instruction and the true random number module of the domestic TEE). The length of the OTP key is exactly the same as the amount of data of the important parameters in the block.

[0045] TEE performs OTP XOR encryption on important parameters in the block to generate encrypted blocks;

[0046] The TEE stores the mapping relationship between "block unique identifier - OTP key" in the TEE's internal secure storage area (including Intel SGX's Enclave Page Cache and the secure storage unit of the domestic TEE) for subsequent decryption calls.

[0047] S6: The GPU receives the encrypted block and restores the encrypted parameters in the encrypted block to the corresponding parameter positions of the original block to form a semi-encrypted block; the GPU computing unit is called to perform parallel linear ciphertext operation on the semi-encrypted block to generate a ciphertext operation result block with a unique block identifier.

[0048] The process of restoring the encrypted parameters to their corresponding positions in the original block includes:

[0049] The GPU associates the corresponding original block with non-critical parameters in the plaintext buffer based on the unique identifier of the encrypted block.

[0050] By using the important parameter coordinates (row number and column number in the original block) recorded in the block parameter package, the encrypted important parameters are accurately backfilled to the corresponding coordinate positions in the original block, forming a "semi-encrypted block" (encrypted parameters + non-important plaintext parameters).

[0051] The GPU counts the number of encrypted parameters after backfilling and compares it with the total number of important parameters for the block selected in step S2. If the counts are inconsistent, a "retransmission request for encrypted block" is sent to the TEE to ensure the integrity of the parameter restoration.

[0052] S7: The GPU sends the ciphertext operation results in blocks to the TEE. The TEE decrypts the ciphertext operation results in blocks and performs non-linear operations on the decrypted results. Then, a preset encryption algorithm is used to encrypt the non-linear operation results a second time, and the second encrypted results are sent back to the GPU.

[0053] The GPU sends the ciphertext operation results in blocks to the TEE. The TEE uses the corresponding encryption key to decrypt the ciphertext operation result blocks, performs nonlinear operations on the decrypted results, and then uses a preset encryption algorithm to encrypt the nonlinear operation results a second time, generating a second-encrypted block that is sent back to the GPU. The preset encryption algorithm is the OTP encryption algorithm, generated in real-time by a hardware random number generator within the TEE (such as the RDRAND instruction of SGX or TRNG of TDX). Each block corresponds to an independent key (avoiding cross-block reuse and meeting the core requirement of "one-time key"). The length requirement is exactly the same as the length of the "nonlinear operation result block" to be encrypted (if the block is 1024-dimensional float32 data, the key is also a 1024-dimensional float32 random sequence, ensuring that each data element has a unique encryption factor).

[0054] S8: The GPU performs linear operations on the secondary encryption result, passes the result to the TEE for decryption, and continues until the final inference result is obtained.

[0055] The following is an example of the present invention:

[0056] Model type: Large-scale medical image diagnosis model with hundreds of billions of parameters (based on Vision Transformer architecture).

[0057] Input: 224×224×3 RGB medical image (CT lung scan);

[0058] Output: Probabilities of three outcomes (lung cancer, benign nodule, normal);

[0059] Model weights: 80GB (FP16 precision, stored on NVMe SSD).

[0060] Step 1: Deployment and Verification of Computation

[0061] By traversing the 12 attention layers of the model using PyTorch FX, “attention score calculation, Q / K linear transformation” is marked as GPU operation, and “GeLU activation, attention weight Softmax” is marked as TEE operation;

[0062] Step 2: Filtering Important Parameters

[0063] The CT images were processed by ResNet50 to extract 2048×1 feature vectors, which were then normalized to the [0,1] interval by the GPU. The Q matrix (2048×64) and K matrix (2048×64) were then calculated.

[0064] Calculate the Score matrix (2048×2048), with mean μ=0.42, standard deviation σ=0.15, and threshold.

[0065] T = 0.42 + 1.2 × 0.15 = 0.6, selecting 128 high-contribution elements; reverse positioning of the i-th element of the Q matrix.

[0066] The 128 important parameters of the row (e.g., i=32,105,210) and the j-th column of the K matrix (e.g., j=15,28,70);

[0067] Step 3: Divide into blocks

[0068] Based on the 16×16 tensor core of NVIDIA A100, the Q matrix is ​​divided into (2048 / 16)×(64 / 16)=128×4=512 blocks, labeled as “Q-001-01” to “Q-512-12” (“Q - block number - layer number”).

[0069] Step 4: Transmission of segmented parameter packets

[0070] Taking the "Q-001-01" block (containing 16 important parameters) as an example, the "block parameter package" is constructed as follows: 16-byte identifier: "Q-001-01" (ASCII encoding); 64-byte important parameter data: 16 parameters × 4 bytes (float32); 32-byte SHA-256 checksum: 0x3F7A12CD56E89B0145F23C7D98A0B1C2...; transmitted to TEE through PCIe 5.0 channel, with the channel SM4 encryption key being 0x7A3FCD1256E89B01.

[0071] Step 5: TEE OTP Encryption

[0072] TEE verifies the hash value of the "block parameter packet" to generate a 64-byte OTP key (0xCD3F7A1256E89B01...), and stores the "Q-001-01-0xCD3F..." mapping relationship;

[0073] Perform OTP XOR encryption on 16 key parameters (e.g., parameter 0.7 → 0.7 XOR 0xCD = 0x4A); return the "encrypted block" (16-byte identifier + 64-byte encryption parameters) to the GPU.

[0074] Step 6: Parameter Restoration and Ciphertext Calculation

[0075] The GPU associates the original block with the identifier "Q-001-01", fills the encryption parameters back into the coordinates (such as (3,5), (7,2)), and verifies that the 16 encryption parameters are correct; it calls the CUDA core to perform matrix multiplication on the "semi-encrypted block", generates a 512-byte ciphertext result block, and passes it to the TEE.

[0076] Step 7: TEE decryption and secondary encryption

[0077] TEE is decrypted using the OTP key and then a softmax operation is performed.

[0078] The softmax calculation result is then encrypted using OTP and sent back to the GPU as a "secondary encrypted block".

[0079] Step 8: Decrypt after GPU linear computation and result output.

[0080] The GPU obtains the nonlinear result after OTP encryption, and directly puts the result into linear calculation, such as the local V matrix, for further calculation. Then it is passed to TEE for further decryption until the final inference result is obtained.

[0081] In summary, the core of this invention lies in constructing a large-scale model security deployment system that integrates GPU and TEE, precisely defining computational responsibilities and security boundaries: computationally intensive linear operations (such as attention score matrix calculation, block matrix multiplication, and Q / K / V matrix linear transformation) are encrypted and deployed in the GPU environment, relying on the parallel computing capabilities of the GPU to ensure efficiency; nonlinear operations involving sensitive logic (such as ReLU / GeLU activation and attention weight normalization) are deployed in the TEE environment, leveraging the hardware-level protection of the TEE to ensure data security, thus achieving a balance between "computational efficiency" and "privacy protection" from the underlying architecture.

[0082] To further address the industry pain point of "the contradiction between secure encryption and inference efficiency," this invention innovatively designs a parameter processing mechanism of "blocking + partial encryption":

[0083] At the block level: For key input matrices such as the Q matrix and K matrix, the block size is set to an integer multiple of the GPU tensor core computation granularity (including 16×16 for NVIDIA GPUs and 32×32 for AMD GPUs) to ensure that the blocks are adapted to the GPU parallel computing units.

[0084] Encryption level: For the sub-blocks after the block is divided, encryption is only performed on the "sensitive areas located by the reverse positioning of the attention score matrix" (such as the parameters corresponding to high contribution elements), while plaintext is left in non-sensitive areas. At the same time, a simplified OTP key that is completely matched with the amount of data in the sensitive areas is generated (the key length is reduced by 60%-80% compared to full block encryption), which greatly reduces the overhead of key generation and storage.

[0085] At the computational level: The GPU directly performs linear operations on a hybrid sub-block of "encrypted sensitive area + plaintext non-sensitive area" and outputs a hybrid result of part of the ciphertext; after receiving the TEE, it only targets the ciphertext area in the decryption result to avoid the latency loss caused by full data decryption.

[0086] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A highly efficient full-process encryption method for large models, characterized in that, Includes the following steps: S1: Traverse the operation types of each layer of the large model and deploy the linear operation part on the GPU and the non-linear operation part on the TEE; S2: The large model receives user input data, and the GPU calculates the Q matrix and K matrix based on the user input data. The attention score matrix is ​​generated by multiplying the Q matrix and K matrix. High contribution elements in the attention score matrix are selected, and important parameters in the Q matrix and K matrix are located in reverse based on the high contribution elements. S3: Divide the Q matrix and K matrix into blocks and assign a unique identifier to each block; S4: The GPU extracts important parameters from each block and constructs a block parameter packet with a unique block identifier and check value, which is then transmitted to the TEE through the PCIe secure channel. Non-critical parameters are stored in the GPU plaintext cache. S5: The TEE verifies the checksum of the block parameter packet. After the verification is successful, it performs encryption processing on the important parameters in the block to generate an encrypted block. The encrypted block is then sent back to the GPU through the PCIe secure channel. S6: The GPU receives the encrypted block and restores the encrypted parameters in the encrypted block to the corresponding parameter positions of the original block, forming a semi-encrypted block; The GPU computing unit is invoked to perform parallel linear ciphertext operations on the semi-encrypted blocks, generating ciphertext operation result blocks with unique block identifiers; S7: The GPU sends the ciphertext operation results in blocks to the TEE. The TEE decrypts the ciphertext operation results in blocks and performs non-linear operations on the decrypted results. Then, it uses a preset encryption algorithm to encrypt the non-linear operation results a second time and generates a second-encrypted result to send back to the GPU. S8: The GPU performs linear operations on the secondary encryption result, passes the result to the TEE for decryption, and continues until the final inference result is obtained.

2. The efficient large-scale model full-process encryption method according to claim 1, characterized in that, Step S2, the process of filtering high-contribution elements in the attention score matrix and using these high-contribution elements to deduce important parameters in the Q and K matrices, includes: Calculate the mean μ and standard deviation σ of the elements in the attention score matrix, set a threshold T = μ + 1.2 × σ, and filter out high contribution elements with values ​​≥ T in the score matrix; based on the coordinates (row i, column j) of the high contribution element in the score matrix, reverse locate the parameter corresponding to it in the i-th row of the Q matrix and the j-th column of the K matrix, and use this parameter as the important parameter.

3. The efficient large-scale model full-process encryption method according to claim 1, characterized in that, In step S3, when dividing the Q matrix and K matrix into blocks, the block size is an integer multiple of the GPU tensor core computation granularity.

4. The efficient large-scale model full-process encryption method according to claim 1, characterized in that, Step S5, the process of encrypting important parameters in the block, includes: TEE uses the OTP encryption algorithm and generates the OTP key through a hardware random number generator. The length of the OTP key is exactly the same as the amount of data of the important parameters in the block. TEE performs OTP XOR encryption on important parameters in the block to generate encrypted blocks; The TEE stores the mapping relationship between "block unique identifier - OTP key" in the TEE's internal secure storage area for subsequent decryption calls.

5. The efficient large-scale model full-process encryption method according to claim 1, characterized in that, Step S6, the process of restoring the encrypted parameters to the corresponding parameter positions in the original block, includes: The GPU associates the original block corresponding to the encrypted block with non-critical parameters in the plaintext buffer based on the unique identifier of the encrypted block. Based on the important parameter coordinates recorded in the block parameter package, the encryption parameters are accurately backfilled into the corresponding coordinate positions of the original block to form a semi-encrypted block; The GPU counts the number of encrypted parameters after backfilling and compares it with the total number of important parameters for the block selected in step S2. If the counts are inconsistent, an encrypted block retransmission request is sent to the TEE to ensure the integrity of the parameter restoration.

6. The efficient large-scale model full-process encryption method according to claim 1, characterized in that, In step S7, the preset encryption algorithm is the OTP encryption algorithm.

Images