Big language model parameter encryption protection method and system based on virtualization

Abstract

The invention provides a virtualization-based big language model parameter encryption protection method and system, and the method comprises the steps: carrying out the corresponding preprocessing of a current to-be-protected big language model in a trusted environment, obtaining a modified linear operator parameter, and carrying out the encryption of a nonlinear operator; the non-trusted virtual machine is a main interface which can be accessed by a user, executes a reasoning task on the preprocessed large language model, and when operation of nonlinear operator encryption needs to be carried out, the non-trusted virtual machine is switched to the trusted virtual machine by calling the service of the virtual machine hypervisor, and the trusted virtual machine completes the operation of nonlinear operator encryption; and the trusted virtual machine processes the encryption operator operation unauthorized to be processed by the non-trusted virtual machine, and returns a result to the non-trusted virtual machine for next calculation of the non-trusted virtual machine. According to the method, mathematical characteristics of different operators of the model are identified, a mode of dynamically modifying the weight of the linear operator without influencing the output result of the model is provided, and safe and efficient operation of the linear operator is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of large language model protection, and in particular to a large language model parameter encryption protection method and system based on virtualization. Background Art

[0002] In recent years, with the rapid development of deep learning technology, large language models (LLMs) have gradually become a key research direction in the field of artificial intelligence. Large language models are typically deep neural networks based on the Transformer architecture. Pre-trained on large corpora, they are able to capture the complex semantics and contextual relationships of language, demonstrating excellent performance in tasks such as text generation, natural language understanding, knowledge reasoning, and interactive dialogue.

[0003] Despite significant technological progress in large language models, numerous challenges remain in parameter security and privacy protection. Model parameter leakage or illegal copying can lead to severe economic losses and intellectual property risks. Furthermore, sensitive training data can be maliciously restored or inferred, seriously threatening user privacy. Therefore, researching and developing effective model protection mechanisms, data privacy protection technologies, and security strategies has become a key topic in the large language model field.

[0004] Currently, large language models pose serious parameter and data security risks in practical applications. To address these issues, the industry has proposed the following three types of encryption protection solutions:

[0005] (1) Model file and user input encryption scheme. This scheme includes two methods: static encryption and homomorphic encryption. Static encryption technology, such as the model file static encryption scheme proposed by Apple, encrypts the model file before distribution and uses it in plain text in memory after decryption at runtime. Although it can effectively prevent files from being directly copied, it is difficult to prevent memory data leakage, and the security protection is limited. Homomorphic encryption allows direct calculations on encrypted data, and the data remains encrypted during the calculation process, such as the encrypted inference service provided by the Delphi system. However, homomorphic encryption has a huge computational overhead and is difficult to meet scenarios with high real-time requirements, which limits its widespread application in large-scale models.

[0006] (2) Protection solutions based on hardware security mechanisms. This type of solution mainly achieves security protection through trusted execution environment (TEE) and GPU confidential computing (GPU CC). TEE solutions (such as Intel SGX, ARM TrustZone, and AMD SEV) use hardware isolation technology to ensure that model parameters and inference data run safely in a protected environment. SOTER improves overall performance by placing part of the model inference task outside the TEE and part of it in the TEE for secure computing; StrongBox uses the ARM unified memory architecture to achieve an effective combination of GPU computing and TEE. However, the TEE solution faces problems such as limited computing resources and poor cross-platform compatibility. GPU CC technology (such as NVIDIA NCC) focuses on protecting data security during GPU computing and ensuring that model parameters and data run safely in an untrusted environment. However, GPU CC currently relies on expensive and limited high-end hardware resources (such as H100 and H800), and performance loss and high cost limit its widespread application. Although PipeLLM optimizes the computing process of GPU CC and improves computing performance, the hardware dependency problem remains prominent.

[0007] (3) Model obfuscation protection scheme. Model obfuscation complicates the model's parameters and structure, increases the difficulty of model reverse engineering, and protects the model's parameters and structural information. The MindSpore framework implements a dynamic obfuscation function to ensure that the model structure is in an obfuscated state during runtime, significantly improving security. SOTER further combines TEE and obfuscation technology to improve the security and operational performance of model obfuscation. However, model obfuscation technology itself has the risk of performance overhead and insufficient anti-attack capabilities, and its effect is limited in the face of advanced reverse attacks. Summary of the Invention

[0008] In view of the defects in the prior art, the purpose of the present invention is to provide a large language model parameter encryption protection method and system based on virtualization.

[0009] According to the present invention, a large language model parameter encryption protection method based on virtualization includes:

[0010] Preprocess the large language model that needs to be protected in a trusted environment to obtain modified linear operator parameters and encrypt the nonlinear operator.

[0011] The untrusted virtual machine is the main interface accessible to users. It performs reasoning tasks on the preprocessed large language model and, when non-linear operator encryption operations are required, switches to the trusted virtual machine by calling the service of the virtual machine hypervisor. The trusted virtual machine completes the non-linear operator encryption operations.

[0012] The trusted virtual machine processes the cryptographic operator operations that the untrusted virtual machine is not authorized to process, and returns the results to the untrusted virtual machine for the next calculation of the untrusted virtual machine.

[0013] Preferably, the virtual machine hypervisor provides an API interface to the trusted virtual machine and the non-trusted virtual machine for allocating shared memory between the trusted virtual machine and the non-trusted virtual machine, as well as for transferring parameters and return results;

[0014] Data is transmitted between untrusted virtual machines and trusted virtual machines through shared memory between virtual machines.

[0015] Preferably, the UEFI Secure Boot technology is used to verify the integrity of the virtual machine hypervisor, and then the virtual machine hypervisor verifies the integrity of the trusted software in the trusted virtual machine.

[0016] Preferably, performing corresponding preprocessing on the large language model that currently needs to be protected in the trusted environment includes implementing it through the associative law of linear matrix operations, and the sub-steps are as follows:

[0017] Step S1.1: In a trusted environment, parse the content of the large language model that needs to be protected, including identifying all linear or nonlinear operators;

[0018] Step S1.2: For each linear operator, randomly generate a scale coefficient, then modify the parameters of the linear operator, and save the corresponding operator name and random coefficient value of each linear operator in a new file named encrypted_model;

[0019] Step S1.3: For nonlinear operators, save all their operator names and parameter values ​​in the new file encrypted_model in step S1.2. In the original file, fill the original parameter content with random values ​​and mark the type of this parameter matrix as the newly added ENCRYPTED_TENSOR;

[0020] Step S1.4: The original model file is processed and encrypted_model is encrypted for protection. It needs to be decrypted when used by the trusted virtual machine.

[0021] Preferably, the step S1.2 includes:

[0022] Determine whether the parameters of the current linear operator are quantized. If so, dequantize the current parameters and multiply them by the scale coefficient, then quantize them again and save them in the original model file. If not, directly multiply them by the scale coefficient to modify the parameter data in the file.

[0023] Preferably, the key used for decryption is encrypted and protected once by the virtual machine hypervisor using the TPM hardware, and the decryption service is provided by the virtual machine hypervisor only when the trusted side virtual machine needs to load the encrypted_model and decrypt it.

[0024] Preferably, allocating shared memory between the non-trusted virtual machine and the trusted virtual machine includes the following steps:

[0025] Step S2.1: When the trusted virtual machine starts, the user-mode computing framework in the trusted virtual machine calls the malloc library function to pre-allocate a piece of memory space at startup, and calls register_prealloc_shm_trusted through the kernel module of the trusted virtual machine to save the address of the memory space in a variable named SHM_GPA_PRE_ALLOC in the kernel of the trusted virtual machine. The variable stores the GPA of this pre-allocated memory space in the trusted virtual machine;

[0026] Step S2.2: The trusted virtual machine kernel initiates a hypercall in register_prealloc_shm_trusted and passes SHM_GPA_PRE_ALLOC to the virtual machine hypervisor. The virtual machine hypervisor records the real physical address corresponding to the pre-allocated virtual machine physical address as SHM_PA.

[0027] Step S2.3: When inference starts, the user-mode inference framework in the untrusted virtual machine calls the malloc library function to allocate a block of memory space, and then calls the kernel module interface register_shm_untrusted of the untrusted kernel to request the creation of shared memory with the trusted virtual machine;

[0028] Step S2.4: The kernel module of the untrusted virtual machine initiates a hyper call, passes in the memory address allocated in step S2.3, and switches to the virtual machine hypervisor. The virtual machine hypervisor receives the GPA parameter from the untrusted virtual machine, modifies the page table to map the GPA of the untrusted virtual machine to the SHM_PA recorded in step S2.3, completes the creation of shared memory, and returns the result to the untrusted virtual machine kernel;

[0029] Step S2.5: The non-trusted virtual machine kernel receives the return result and returns the return result to the reasoning framework in the non-trusted virtual machine. The reasoning framework then uses the corresponding shared memory in reasoning.

[0030] Preferably, performing the inference task on the preprocessed large language model includes:

[0031] Step S3.1: The untrusted virtual machine loads the preprocessed model file and transfers the encrypted portion of the model file to the pre-allocated shared memory through the system call provided by the untrusted kernel. The data type in the shared memory is set to ENCRYPTED_MODEL_DATA to inform the trusted side of the data type.

[0032] Step S3.2: When processing the system call in step S3.1, the untrusted kernel calls the hyper call provided by the virtual machine hypervisor to initiate a virtual machine switch; the hypervisor saves the execution context of the untrusted side virtual machine and switches to the trusted side virtual machine. The trusted side kernel returns the execution flow to the trusted side reasoning framework.

[0033] Step S3.3: The worker thread of the trusted-side inference framework notices that new data is in shared memory and its type is ENCRYPTED_MODEL_DATA. It calls the process_model function to parse the model data, then calls the system call provided by the trusted-side kernel. The trusted-side kernel then calls the hyper call to switch virtual machines and finally returns to the untrusted-side inference framework to continue inference.

[0034] Step S3.4: The untrusted virtual machine receives the user's input and starts reasoning.

[0035] Step S3.5: During normal reasoning, the untrusted virtual machine can complete the reasoning independently. However, when encountering a nonlinear operator marked as encrypted, the untrusted virtual machine cannot handle it independently and requires assistance from the trusted side.

[0036] Step S3.6: The trusted side virtual machine framework starts executing, finds data of type INPUT_DATA in the shared memory, and starts computing. The untrusted side reasoning framework obtains the output result from the shared memory.

[0037] Repeat steps S3.4 to S3.6 until one inference is completed.

[0038] Preferably, when encountering a nonlinear operator marked as encrypted, the untrusted side virtual machine cannot process it independently and requires assistance from the trusted side, including: the untrusted side reasoning framework first copies the input of the current nonlinear operator, that is, the output of the previous operator, to the shared memory, and sets the data type of the shared memory to INPUT_DATA. Subsequently, the untrusted side virtual machine calls the system call, and then the kernel calls the hyper call. The hypervisor completes the saving and switching of the virtual machine context and enters the trusted side virtual machine to run.

[0039] The calculation in step S3.6 includes the trusted side virtual machine first multiplying all the coefficients since the last time it entered the trusted side, and then dividing the input data in the shared memory by the total coefficient after the multiplication to restore the input data. The trusted side virtual machine then operates on the restored input, the corresponding nonlinear operator parameters and the corresponding operator to obtain an output. The trusted side virtual machine places the output in the shared memory and sets the data type to DATA_OUTPUT. The trusted side reasoning framework also switches back to the non-trusted side virtual machine by calling the system call and then the trusted side kernel calls the hyper call.

[0040] According to the present invention, a large language model parameter encryption protection system based on virtualization includes: an untrusted virtual machine, a trusted virtual machine and a virtual machine hypervisor;

[0041] Preprocess the large language model that needs to be protected in a trusted environment to obtain modified linear operator parameters and encrypt the nonlinear operator.

[0042] The untrusted virtual machine is the main interface accessible to users. It performs reasoning tasks on the preprocessed large language model and, when non-linear operator encryption operations are required, switches to the trusted virtual machine by calling the service of the virtual machine hypervisor. The trusted virtual machine completes the non-linear operator encryption operations.

[0043] The trusted virtual machine processes the cryptographic operator operations that the untrusted virtual machine is not authorized to process, and returns the results to the untrusted virtual machine for the next calculation of the untrusted virtual machine.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. The present invention uses the UEFI secure boot function and hardware virtualization technology to construct a secure isolated execution environment, so that some operators that need to be calculated in the trusted virtual machine can be safely stored in the trusted virtual machine memory and will not be stolen by the untrusted virtual machine, ensuring the security of the system during startup.

[0046] 2. This invention identifies the mathematical characteristics of different operators in the model and proposes a method to dynamically modify the weights of linear operators without affecting the model output results, thereby ensuring the safe and efficient operation of linear operators.

[0047] 3. The present invention utilizes the associative property of linear operators during reasoning in a large language model to modify the linear operators, thereby allowing the parameters related to the linear operators to run safely in an untrusted virtual machine without leaking any information about the linear operators. Virtualization technology is used to securely protect part of the computing process, while the output of the linear operators can be securely restored.

[0048] 4. The present invention uses shared memory technology to efficiently transmit data between trusted and non-trusted virtual machines, effectively reducing performance overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0050] Figure 1 Schematic diagram of the system architecture of the present invention;

[0051] Figure 2 A schematic diagram of a shared memory allocation and registration process between a non-trusted virtual machine and a trusted virtual machine in the present invention;

[0052] Figure 3 This is a flow chart of the non-trusted virtual machine of the present invention performing reasoning tasks on a pre-processed large language model. DETAILED DESCRIPTION

[0053] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0054] The present invention uses a more common and easily accessible hardware isolation mechanism, namely virtualization technology, combined with the characteristics of linear parameters in large model parameters in calculations, and calculates the reasoning of large models in untrusted virtual machines and trusted virtual machines respectively, thereby ensuring the security of model parameters while taking into account the performance of model reasoning, and without using hardware features that are limited in access or insufficiently popular.

[0055] Example 1

[0056] According to the present invention, a large language model parameter encryption protection method based on virtualization is provided. Figure 1As shown, it includes: performing corresponding preprocessing on the large language model that currently needs to be protected in a trusted environment to obtain modified linear operator parameters and encrypt the nonlinear operator; the non-trusted virtual machine is the main interface accessible to users, performs reasoning tasks on the pre-processed large language model, and when non-linear operator encryption operations are required, switches to the trusted virtual machine by calling the virtual machine hypervisor service, and the trusted virtual machine completes the non-linear operator encryption operations. The trusted virtual machine processes the encryption operator operations that the non-trusted virtual machine is not authorized to process, and returns the results to the non-trusted virtual machine for the next calculation of the non-trusted virtual machine. The virtual machine hypervisor is used for the startup, operation, memory isolation and virtual machine switching of the trusted virtual machine and the non-trusted virtual machine. The virtual machine hypervisor provides an API interface to the trusted virtual machine and the non-trusted virtual machine for allocating shared memory between the trusted virtual machine and the non-trusted virtual machine, as well as for the transmission of parameters and return results. Data is transmitted between the non-trusted virtual machine and the trusted virtual machine through the shared memory between the virtual machines.

[0057] The trusted base in this invention includes the virtual machine hypervisor and the software in the trusted virtual machine. The Linux kernel and reasoning framework running in the untrusted virtual machine are untrusted. This invention ensures that the trusted software has not been modified by using UEFI's Secure Boot technology to verify the integrity of the virtual machine hypervisor. The virtual machine hypervisor then verifies the integrity of the software in the trusted virtual machine.

[0058] In a trusted environment, the large language model that needs to be protected is preprocessed accordingly, including using the associative law of linear matrix operations to ensure that the weights of all linear operators (such as matrix multiplication) comply with the associative law of linear matrix operations. For example,

[0059] Op1o×Op1μ=Op1m,

[0060] input×Op1o=output,

[0061] Op1m×input=output ′ ,

[0062]

[0063] The above example describes how to modify the original operator Op1o using the associative law of linear matrix operations to obtain the modified operator Op1m, and the output output obtained by the operator Op1m and the input input ′Calculate the output of the original operator. By storing the operator Op1μ in a trusted virtual machine, the plaintext Op1m can be safely placed in a non-trusted virtual machine for efficient computation. Most nonlinear operators do not conform to the associative law described above, and therefore cannot be computed on the non-secure side after modification. The present invention handles such operators by encrypting and marking them. The non-trusted virtual machine then completes the computation of these operators by requesting services from the trusted virtual machine.

[0064] The trusted virtual machine contains a decrypted nonlinear operator. Before performing calculations, the trusted virtual machine must first use the associative property to restore the output of the previous step passed to the untrusted virtual machine. It then operates on the parameters of the nonlinear operator and returns the result to the untrusted virtual machine. This ensures the confidentiality of the parameters and, in turn, the security of the model.

[0065] Furthermore, the corresponding preprocessing of the large language model that currently needs to be protected in the trusted environment includes the following sub-steps:

[0066] Step S1.1: In a trusted environment, parse the content of the large language model that currently needs to be protected, including identifying all linear operators or nonlinear operators.

[0067] Step S1.2: For each linear operator, randomly generate a scale coefficient, then modify the parameters of the linear operator. For each linear operator, save the corresponding operator name and random coefficient value in a new file named encrypted_model.

[0068] The step S1.2 includes: determining whether the parameters of the current linear operator are quantized; if so, dequantizing the current parameters and multiplying them by the scale coefficient, then requantizing them and saving them in the original model file, i.e., the large language model file that needs to be protected, wherein the model file contains a specific file format and model parameters; if not, directly multiplying them by the scale coefficient to modify the parameter data in the file. The modified parameters of the linear operator are directly saved in the original file, overwriting the original unmodified parameter data, while reusing the original model file format to facilitate direct recognition and use by the inference framework.

[0069] Step S1.3: For nonlinear operators, save their operator names and parameter values ​​in the new file encrypted_model in step S1.2. In the original file, use random values ​​to fill the original parameter content, and mark the type of this parameter matrix as the newly added ENCRYPTED_TENSOR. For example, in the original large language model file, the metadata of each operator is stored in the front part of the file, describing the name, position in the file, length, and type of the operator (such as the parameters of the softmax operator / matmul operator. The former is nonlinear, so it can be known that this operator needs to be encrypted). For example, an operator may start at the 1000th byte of the file and end at the 1200th byte of the file. Through the file reading and writing interface, the original content, such as 0x12345678, is overwritten with random content such as 0x43786836, so as not to expose the original parameters.

[0070] Step S1.4: After the original model file is processed, the encrypted_model is encrypted for protection. It needs to be decrypted when used by the trusted virtual machine. The decryption key is encrypted once by the virtual machine hypervisor using the TPM hardware. The decryption service is provided by the virtual machine hypervisor only when the trusted side virtual machine needs to load the encrypted_model and decrypt it.

[0071] In Type 1 virtualization (also known as bare-metal virtualization), the hypervisor runs directly on the physical hardware and is responsible for allocating and managing resources for each client (virtual machine). Address isolation between two clients is achieved through the following mechanism: each client has an independent virtual memory space, and the hypervisor dynamically maps their guest physical addresses (GPA) to different host physical memory (HPA) areas. For example, when client A accesses its guest physical address 0x1000, the hypervisor may point it to the actual physical address 0xA000, while client B's guest physical address 0x1000 is mapped to physical address 0xB000. This hardware memory management unit (MMU) combined with page table translation technology ensures that the two virtual machines cannot directly access each other's memory space, thereby achieving strict memory isolation and ensuring system security and stability.

[0072] Since data needs to be transmitted between the untrusted virtual machine and the trusted virtual machine in the present invention, and the data is large in scale, an efficient transmission method is required. Therefore, the present invention adopts shared memory between virtual machines to carry out message transmission. Figure 2 As shown, the shared memory allocation between the untrusted virtual machine and the trusted virtual machine includes the following steps:

[0073] Step S2.1: When the trusted virtual machine is started, the user-mode computing framework in the trusted virtual machine calls the malloc library function to pre-allocate a piece of memory space at startup, and calls register_prealloc_shm_trusted through the kernel module of the trusted virtual machine to save the address of the memory space in a variable named SHM_GPA_PRE_ALLOC in the kernel of the trusted virtual machine. The variable saves the GPA (client physical address) of this pre-allocated memory space in the trusted virtual machine.

[0074] Step S2.2: The trusted virtual machine kernel initiates a hypercall in register_prealloc_shm_trusted and passes SHM_GPA_PRE_ALLOC to the virtual machine hypervisor. The virtual machine hypervisor records the real physical address corresponding to the pre-allocated virtual machine physical address as SHM_PA.

[0075] Step S2.3: When reasoning starts, the user-mode reasoning framework in the untrusted virtual machine calls the malloc library function to allocate a piece of memory space, and then calls the kernel module interface register_shm_untrusted of the untrusted kernel to apply for the creation of shared memory with the trusted virtual machine.

[0076] Step S2.4: The kernel module of the non-trusted virtual machine initiates a hyper call, passes in the memory address allocated in step S2.3, and switches to the virtual machine hypervisor. The virtual machine hypervisor receives the GPA (guest physical address) parameter from the non-trusted virtual machine, and maps the GPA of the non-trusted virtual machine to the SHM_PA recorded in step S2.3 by modifying the page table, completing the creation of shared memory and returning the result to the non-trusted virtual machine kernel.

[0077] Step S2.5: The non-trusted virtual machine kernel receives the return result and returns the return result to the reasoning framework in the non-trusted virtual machine. The reasoning framework then uses the corresponding shared memory in reasoning.

[0078] like Figure 3 As shown in the figure, after completing model preprocessing and shared memory allocation, the main process of an inference includes:

[0079] Step S3.1: The untrusted side virtual machine loads the preprocessed model file, and at the same time transfers the encrypted part of the model file (including the modification of the coefficients of each linear operator and the encrypted nonlinear operator) to the pre-allocated shared memory through the system call provided by the untrusted kernel, and sets the data type in the shared memory to ENCRYPTED_MODEL_DATA to inform the trusted side of the data type.

[0080] Step S3.2: When processing the system call in step S3.1, the untrusted kernel invokes the hypercall provided by the virtual machine hypervisor, initiating a virtual machine switch. The hypervisor saves the execution context of the untrusted virtual machine and switches to the trusted virtual machine. The trusted kernel then returns execution flow to the trusted inference framework.

[0081] Step S3.3: The trusted-side inference framework's worker thread notices the presence of new data in shared memory, of type ENCRYPTED_MODEL_DATA. It calls the process_model function to parse the model data, including decryption and constructing a mapping table from operator names to coefficients. It then calls a system call provided by the trusted-side kernel, which then calls a hyper call to switch virtual machines, ultimately returning to the untrusted-side inference framework to continue inference.

[0082] Step S3.4: The untrusted virtual machine receives the user's input and starts reasoning. During normal reasoning, the untrusted virtual machine can complete the reasoning independently.

[0083] Step S3.5: When encountering a nonlinear operator marked as encrypted, the untrusted VM cannot handle it independently and requires assistance from the trusted side. The untrusted inference framework first copies the input of the current nonlinear operator (i.e., the output of the previous operator) to shared memory and sets the shared memory data type to INPUT_DATA. The untrusted VM then invokes a system call, which in turn invokes a hypercall. The hypervisor saves and switches the VM context, and the trusted VM enters execution.

[0084] Step S3.6: The trusted side virtual machine framework starts executing, and when it finds data of type INPUT_DATA in the shared memory, it starts calculating. The trusted side virtual machine first restores the input data by multiplying all the coefficients since the last time it entered the trusted side, and then divides the input data in the shared memory by the total coefficient after the multiplication. The trusted side virtual machine then calculates the restored input, the corresponding nonlinear operator parameters, and the corresponding operator to obtain an output. To ensure security, this output also needs to be scaled by a certain coefficient, and this coefficient needs to be saved. After completion, the trusted side virtual machine places this output in the shared memory and sets the data type to DATA_OUTPUT. The trusted side reasoning framework also switches back to the non-trusted side virtual machine by calling a system call, and then the trusted side kernel calls a hyper call.

[0085] Step S3.7: The untrusted side reasoning framework obtains the output result from the shared memory and continues to repeat steps S3.4 to S3.6 until one reasoning is completed.

[0086] The purpose of this invention is to ensure that parameters of large language models are protected even when they are deployed locally. This addresses several issues with existing solutions, including insufficient security, poor performance, and reliance on hard-to-access hardware. Furthermore, isolation using virtualization technology presents difficulties in dynamically partitioning hardware resources. A partitioning method more suitable for large model scenarios is needed.

[0087] Example 2

[0088] The present invention also provides a large language model parameter encryption protection system based on virtualization. The large language model parameter encryption protection system based on virtualization can be implemented by executing the process steps of the large language model parameter encryption protection method based on virtualization. That is, those skilled in the art can understand the large language model parameter encryption protection method based on virtualization as a preferred implementation of the large language model parameter encryption protection system based on virtualization.

[0089] According to the present invention, a large language model parameter encryption protection system based on virtualization is provided, comprising:

[0090] In a trusted environment, the large language model that currently needs to be protected is preprocessed accordingly to obtain the modified linear operator parameters and encrypt the nonlinear operator. The corresponding preprocessing of the large language model that currently needs to be protected in a trusted environment includes implementation through the associative law of linear matrix operations, and the sub-steps are as follows: Module M1.1: In a trusted environment, the content of the large language model that currently needs to be protected is parsed, including identifying all linear operators or nonlinear operators. Module M1.2: For the linear operator, a scale coefficient is randomly generated, and then the parameters of the linear operator are modified, and for each linear operator, the corresponding operator name and the value of the random coefficient are additionally saved in a new file named encrypted_model. The module M1.2 includes: determining whether the parameters of the current linear operator are quantized. If so, the current parameters are dequantized and multiplied by the scale coefficient, and then quantized again and saved in the original model file. If not, the parameters are directly multiplied by the scale coefficient to modify the parameter data in the file. Module M1.3: For nonlinear operators, all operator names and parameter values ​​are saved in the new file encrypted_model in module M1.2. In the original file, the original parameter content is filled with random values, and the type of this parameter matrix is ​​marked as the newly added ENCRYPTED_TENSOR. Module M1.4: After the original model file is processed, encrypted_model is encrypted for protection and needs to be decrypted when used by the trusted virtual machine. The decryption key is encrypted once by the virtual machine hypervisor using the TPM hardware. The virtual machine hypervisor provides decryption services only when the trusted side virtual machine needs to load encrypted_model and decrypt it.

[0091] The non-trusted virtual machine is the primary user-accessible interface, performing inference tasks on the pre-processed large language model. When non-linear operator encryption operations are required, the trusted virtual machine is switched to by invoking the hypervisor service of the virtual machine, which then completes the non-linear operator encryption operations. The trusted virtual machine processes encryption operator operations that the non-trusted virtual machine is not authorized to handle and returns the results to the non-trusted virtual machine for further calculations. Performing inference tasks on the pre-processed large language model includes:

[0092] Module M3.1: The untrusted virtual machine loads the preprocessed model file and transfers the encrypted part of the model file to the pre-allocated shared memory through the system call provided by the untrusted kernel. The data type in the shared memory is set to ENCRYPTED_MODEL_DATA to inform the trusted side of the data type.

[0093] Module M3.2: When the untrusted kernel processes the system call in module M3.1, it invokes the hypercall provided by the virtual machine hypervisor, initiating a virtual machine switch. The hypervisor saves the execution context of the untrusted virtual machine and switches to the trusted virtual machine. The trusted kernel then returns execution flow to the trusted inference framework.

[0094] Module M3.3: The worker thread of the trusted-side inference framework notices that there is new data in the shared memory, and its type is ENCRYPTED_MODEL_DATA. It calls the process_model function to parse the model data, then calls the system call provided by the trusted-side kernel. The trusted-side kernel then calls the hyper call to switch the virtual machine and finally returns to the untrusted-side inference framework to continue inference.

[0095] Module M3.4: The non-trusted virtual machine receives the user's input and starts reasoning.

[0096] Module M3.5: During normal reasoning, the untrusted virtual machine can complete the inference independently. However, when encountering a nonlinear operator marked as encrypted, the untrusted virtual machine cannot handle it independently and requires assistance from the trusted side. The trusted side assistance includes: the untrusted side reasoning framework first copies the input of the current nonlinear operator, that is, the output of the previous operator, to shared memory and sets the data type of the shared memory to INPUT_DATA. The untrusted virtual machine then calls a system call, which is then called by the kernel. The hypervisor saves and switches the virtual machine context, and the trusted side virtual machine enters the running state.

[0097] Module M3.6: The trusted-side virtual machine framework begins execution and discovers data of type INPUT_DATA in shared memory. Calculations begin, and the untrusted-side reasoning framework obtains the output from shared memory. The calculations in module M3.6 involve the trusted-side virtual machine first multiplying all coefficients since the last time it entered the trusted side, then dividing the input data in shared memory by the total multiplied coefficients to restore the input data. The trusted-side virtual machine then calculates the restored input, the corresponding nonlinear operator parameters, and the corresponding operator to obtain an output. The trusted-side virtual machine places the output in shared memory and sets the data type to DATA_OUTPUT. The trusted-side reasoning framework also switches back to the untrusted-side virtual machine by invoking a system call, which is then followed by a hyper call by the trusted-side kernel. Modules M3.4 to S3.6 are repeated until a reasoning cycle is complete.

[0098] The virtual machine hypervisor provides an API interface to the trusted virtual machine and the non-trusted virtual machine for allocating shared memory between the trusted virtual machine and the non-trusted virtual machine, as well as the transmission of parameters and return results. Data is transmitted between the non-trusted virtual machine and the trusted virtual machine through the shared memory between the virtual machines. The integrity of the virtual machine hypervisor is verified by using the Secure Boot technology of UEFI, and then the integrity of the trusted software in the trusted virtual machine is verified by the virtual machine hypervisor. The shared memory allocation between the non-trusted virtual machine and the trusted virtual machine includes the following steps: Module M2.1: When the trusted virtual machine is started, the user-mode computing framework in the trusted virtual machine calls the malloc library function at startup to pre-allocate a piece of memory space, and calls register_prealloc_shm_trusted through the kernel module of the trusted virtual machine to save the address of the memory space in the trusted virtual machine kernel in a variable named SHM_GPA_PRE_ALLOC, which stores the GPA of this pre-allocated memory space in the trusted virtual machine. Module M2.2: The trusted VM kernel initiates a hyper call in register_prealloc_shm_trusted, passing SHM_GPA_PRE_ALLOC to the VM hypervisor. The VM hypervisor records the real physical address corresponding to the preallocated VM physical address as SHM_PA. Module M2.3: At the start of inference, the user-mode inference framework in the untrusted VM calls the malloc library function to allocate a block of memory space, then calls the untrusted kernel's kernel module interface register_shm_untrusted to request the creation of shared memory with the trusted VM. Module M2.4: The untrusted VM's kernel module initiates a hyper call, passing in the memory address allocated in module M2.3, and switches to the VM hypervisor. The VM hypervisor receives the GPA parameter from the untrusted VM and modifies the page table to map the untrusted VM's GPA to the SHM_PA recorded in module M2.3, completing the creation of shared memory and returning the result to the untrusted VM kernel. Module M2.5: The non-trusted virtual machine kernel receives the return result and returns the return result to the inference framework in the non-trusted virtual machine. The inference framework then uses the corresponding shared memory in inference.

[0099] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0100] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A large language model parameter encryption protection method based on virtualization, characterized in that: include: Preprocess the large language model that needs to be protected in a trusted environment to obtain modified linear operator parameters and encrypt the nonlinear operator. The untrusted virtual machine is the main interface accessible to users. It performs reasoning tasks on the preprocessed large language model and, when non-linear operator encryption operations are required, switches to the trusted virtual machine by calling the service of the virtual machine hypervisor. The trusted virtual machine completes the non-linear operator encryption operations. The trusted virtual machine processes the cryptographic operator operations that the untrusted virtual machine is not authorized to process, and returns the results to the untrusted virtual machine for the next calculation of the untrusted virtual machine.

2. The method for encrypting and protecting large language model parameters based on virtualization according to claim 1, characterized in that: The virtual machine hypervisor provides an API interface to the trusted virtual machine and the non-trusted virtual machine for allocating shared memory between the trusted virtual machine and the non-trusted virtual machine, as well as transferring parameters and return results; Data is transmitted between untrusted virtual machines and trusted virtual machines through shared memory between virtual machines.

3. The method for encrypting and protecting large language model parameters based on virtualization according to claim 1, characterized in that: Utilize UEFI's Secure Boot technology to verify the integrity of the virtual machine hypervisor, which then verifies the integrity of the trusted software in the trusted virtual machine.

4. The method for encrypting and protecting large language model parameters based on virtualization according to claim 1, characterized in that: Preprocessing the large language model that needs to be protected in a trusted environment involves the associative law of linear matrix operations. The sub-steps are as follows: Step S1.1: In a trusted environment, parse the content of the large language model that needs to be protected, including identifying all linear or nonlinear operators; Step S1.2: For each linear operator, randomly generate a scale coefficient, then modify the parameters of the linear operator, and save the corresponding operator name and random coefficient value of each linear operator in a new file named encrypted_model; Step S1.3: For nonlinear operators, save all their operator names and parameter values ​​in the new file encrypted_model in step S1.

2. In the original file, fill the original parameter content with random values ​​and mark the type of this parameter matrix as the newly added ENCRYPTED_TENSOR; Step S1.4: The original model file is processed and encrypted_model is encrypted for protection. It needs to be decrypted when used by the trusted virtual machine.

5. The method for encrypting and protecting parameters of a large language model based on virtualization according to claim 4, characterized in that: The step S1.2 includes: Determine whether the parameters of the current linear operator are quantized. If so, dequantize the current parameters and multiply them by the scale coefficient, then quantize them again and save them in the original model file. If not, directly multiply them by the scale coefficient to modify the parameter data in the file.

6. The method for encrypting and protecting parameters of a large language model based on virtualization according to claim 4, characterized in that: The key used for decryption is encrypted once by the virtual machine hypervisor using the TPM hardware. The decryption service is provided by the virtual machine hypervisor only when the trusted side virtual machine needs to load the encrypted_model and decrypt it.

7. The method for encrypting and protecting parameters of a large language model based on virtualization according to claim 2, characterized in that: The shared memory allocation between untrusted VMs and trusted VMs includes the following steps: Step S2.1: When the trusted virtual machine starts, the user-mode computing framework in the trusted virtual machine calls the malloc library function to pre-allocate a piece of memory space at startup, and calls register_prealloc_shm_trusted through the kernel module of the trusted virtual machine to save the address of the memory space in a variable named SHM_GPA_PRE_ALLOC in the kernel of the trusted virtual machine. The variable stores the GPA of this pre-allocated memory space in the trusted virtual machine; Step S2.2: The trusted virtual machine kernel initiates a hyper call in register_prealloc_shm_trusted and passes SHM_GPA_PRE_ALLOC to the virtual machine hypervisor. The virtual machine hypervisor records the real physical address corresponding to the pre-allocated virtual machine physical address as SHM_PA. Step S2.3: When inference starts, the user-mode inference framework in the untrusted virtual machine calls the malloc library function to allocate a block of memory space, and then calls the kernel module interface register_shm_untrusted of the untrusted kernel to request the creation of shared memory with the trusted virtual machine; Step S2.4: The kernel module of the untrusted virtual machine initiates a hyper call, passes in the memory address allocated in step S2.3, and switches to the virtual machine hypervisor. The virtual machine hypervisor receives the GPA parameter from the untrusted virtual machine, modifies the page table to map the GPA of the untrusted virtual machine to the SHM_PA recorded in step S2.3, completes the creation of shared memory, and returns the result to the untrusted virtual machine kernel; Step S2.5: The non-trusted virtual machine kernel receives the return result and returns the return result to the reasoning framework in the non-trusted virtual machine. The reasoning framework then uses the corresponding shared memory in reasoning.

8. The method for encrypting and protecting parameters of a large language model based on virtualization according to claim 1, characterized in that: Performing reasoning tasks on the pre-processed large language model includes: Step S3.1: The untrusted virtual machine loads the preprocessed model file and transfers the encrypted portion of the model file to the pre-allocated shared memory through the system call provided by the untrusted kernel. The data type in the shared memory is set to ENCRYPTED_MODEL_DATA to inform the trusted side of the data type. Step S3.2: When processing the system call in step S3.1, the untrusted kernel calls the hyper call provided by the virtual machine hypervisor to initiate a virtual machine switch; the hypervisor saves the execution context of the untrusted side virtual machine and switches to the trusted side virtual machine. The trusted side kernel returns the execution flow to the trusted side reasoning framework. Step S3.3: The worker thread of the trusted-side inference framework notices that new data is in shared memory and its type is ENCRYPTED_MODEL_DATA. It calls the process_model function to parse the model data, then calls the system call provided by the trusted-side kernel. The trusted-side kernel then calls the hyper call to switch virtual machines and finally returns to the untrusted-side inference framework to continue inference. Step S3.4: The untrusted virtual machine receives the user's input and starts reasoning. Step S3.5: During normal reasoning, the untrusted virtual machine can complete the reasoning independently. However, when encountering a nonlinear operator marked as encrypted, the untrusted virtual machine cannot handle it independently and requires assistance from the trusted side. Step S3.6: The trusted side virtual machine framework starts executing, finds data of type INPUT_DATA in the shared memory, and starts computing. The untrusted side reasoning framework obtains the output result from the shared memory. Repeat steps S3.4 to S3.6 until one inference is completed.

9. The method for encrypting and protecting parameters of a large language model based on virtualization according to claim 8, characterized in that: When encountering a nonlinear operator marked as encrypted, the untrusted side virtual machine cannot process it independently and requires assistance from the trusted side, including: the untrusted side reasoning framework first copies the input of the current nonlinear operator, that is, the output of the previous operator, to the shared memory, and sets the data type of the shared memory to INPUT_DATA. Then the untrusted side virtual machine calls the system call, and the kernel calls the hyper call. The hypervisor completes the saving and switching of the virtual machine context and enters the trusted side virtual machine to run. The calculation in step S3.6 includes the trusted side virtual machine first multiplying all the coefficients since the last time it entered the trusted side, and then dividing the input data in the shared memory by the total coefficient after the multiplication to restore the input data. The trusted side virtual machine then operates on the restored input, the corresponding nonlinear operator parameters and the corresponding operator to obtain an output. The trusted side virtual machine places the output in the shared memory and sets the data type to DATA_OUTPUT. The trusted side reasoning framework also switches back to the non-trusted side virtual machine by calling the system call and then the trusted side kernel calls the hyper call.

10. A large language model parameter encryption protection system based on virtualization, characterized in that: include: Untrusted virtual machines, trusted virtual machines, and virtual machine hypervisors; Preprocess the large language model that needs to be protected in a trusted environment to obtain modified linear operator parameters and encrypt the nonlinear operator. The untrusted virtual machine is the main interface accessible to users. It performs reasoning tasks on the preprocessed large language model and, when non-linear operator encryption operations are required, switches to the trusted virtual machine by calling the service of the virtual machine hypervisor. The trusted virtual machine completes the non-linear operator encryption operations. The trusted virtual machine processes the cryptographic operator operations that the untrusted virtual machine is not authorized to process, and returns the results to the untrusted virtual machine for the next calculation of the untrusted virtual machine.

Images