Large model fine tuning method and device, electronic equipment and storage medium

Abstract

The invention provides a large model fine tuning method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an initial large model, connecting at least one model layer in parallel with a target low-rank matrix in other model layers except a first model layer and a last model layer of the initial large model in the initial large model, taking a first model layer of the initial large model and an initial low-rank matrix, a target low-rank matrix and a last model layer which are connected in parallel with the first model layer as a dense-state domain model, and deploying the dense-state domain model in a trusted execution environment (TEE); and taking other model layers except the first model layer and the last model layer in the initial large model as plaintext domain models, deploying the plaintext domain models in a plaintext computing environment, updating parameters of a target low-rank matrix according to output of the secret state domain model and output of each model layer in the plaintext domain models to obtain a target large model, and storing the target large model in the plaintext computing environment. Therefore, the model is protected, and the fine tuning training efficiency of the model can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of large models, and in particular to a large model fine-tuning method, device, electronic device and storage medium. Background Art

[0002] High-value large models, especially industry large models, often need to be trained or fine-tuned using industry privacy data. However, the data owner often does not have the computing power required for large model training, or requires multiple parties' industry privacy data for joint training. In this case, the only option is to use artificial intelligence (AI) computing centers to gather industry data for centralized training. Since the trained models contain private industry knowledge, this brings up the issue of privacy protection for large models.

[0003] Therefore, how to ensure that the model is not stolen or abused during centralized training has become a key research direction. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the purpose of the present invention is to propose a large model fine-tuning method, device, electronic device and storage medium, thereby ensuring that the model is not stolen during the large model fine-tuning process, thereby achieving protection of the model. Moreover, since the large model is fine-tuned by the joint target low-rank matrix, the model fine-tuning training efficiency can also be improved.

[0006] The first aspect of the present disclosure provides a large model fine-tuning method, the method comprising:

[0007] Acquire an initial large model, wherein the initial large model comprises: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix;

[0008] The first model layer of the initial large model and its parallel initial low-rank matrix, target low-rank matrix and the last model layer are used as the secret domain model, and the secret domain model is deployed in the trusted execution environment TEE;

[0009] The model layers other than the first model layer and the last model layer in the initial large model are used as plaintext domain models, and the plaintext domain models are deployed in a plaintext computing environment;

[0010] According to the output of each model layer in the dense domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain the target large model.

[0011] The large model fine-tuning device provided in the second aspect of the present disclosure includes:

[0012] An acquisition module is used to acquire an initial large model, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix;

[0013] The first deployment module is used to use the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer as a secret domain model, and deploy the secret domain model in the trusted execution environment TEE;

[0014] The second deployment module is used to use the model layers other than the first model layer and the last model layer in the initial large model as plaintext domain models, and deploy the plaintext domain models in a plaintext computing environment;

[0015] The updating module is used to update the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain the target large model.

[0016] The electronic device proposed in the third aspect embodiment of the present disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the large model fine-tuning method proposed in the first aspect embodiment of the present disclosure is implemented.

[0017] The fourth aspect embodiment of the present disclosure proposes a non-temporary computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the large model fine-tuning method proposed in the first aspect embodiment of the present disclosure.

[0018] The fifth aspect embodiment of the present disclosure proposes a computer program product. When the instructions in the computer program product are executed by a processor, the large model fine-tuning method proposed in the first aspect embodiment of the present disclosure is executed.

[0019] The large model fine-tuning method, device, electronic device and storage medium provided by the present disclosure have at least the following beneficial effects: obtaining an initial large model, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and in the initial large model, at least one model layer is connected in parallel with a target low-rank matrix in other model layers except the first model layer and the last model layer of the initial large model, and then the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as secret domain models, and the secret domain model is deployed in a trusted execution environment TEE, and the other model layers except the first model layer and the last model layer in the initial large model are used as plaintext domain models, and the plaintext domain model is deployed in a plaintext computing environment, and the parameters of the target low-rank matrix are updated according to the output of each model layer in the secret domain model and the plaintext domain model to obtain the target large model, thereby ensuring that the model is not stolen during the large model fine-tuning process, thereby achieving protection of the model, and because the large model is fine-tuned in conjunction with the target low-rank matrix, the model fine-tuning training efficiency can also be improved.

[0020] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 It is a flowchart of a large model fine-tuning method proposed in an embodiment of the present disclosure;

[0023] Figure 2 It is a schematic diagram of the training process of the split large model proposed in one embodiment of the present disclosure;

[0024] Figure 3 It is a partial flow chart of model fine-tuning proposed in one embodiment of the present disclosure;

[0025] Figure 4 is a flow chart of a large model fine-tuning method proposed in another embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of a trusted execution environment provided by an embodiment of the present disclosure;

[0027] Figure 6 It is a structural schematic diagram of a large model fine-tuning device proposed in an embodiment of the present disclosure;

[0028] Figure 7A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and are not to be construed as limitations of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications, and equivalents that fall within the spirit and connotation of the appended claims.

[0030] Figure 1 It is a flowchart of a large model fine-tuning method proposed in an embodiment of the present disclosure.

[0031] Among them, it should be noted that the executor of the large model fine-tuning method of this embodiment is a large model fine-tuning device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, and the electronic device can include but is not limited to a terminal, a server, etc. For example, the terminal can be a mobile phone, a handheld computer, etc.

[0032] like Figure 1 As shown, the large model fine-tuning method includes:

[0033] S101: Obtain an initial large model, wherein the initial large model includes: a plurality of model layers connected sequentially, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix.

[0034] Among them, the initial large model is the large model to be fine-tuned. The initial large model can be a neural network model (Transformers) structure of a self-attention mechanism of a neural network model architecture (Decoder Only). For example, the open source large model (Llama2) is a large model with 32 Decoder layers, and there is no restriction on this.

[0035] The initial large model may include multiple model layers connected in sequence, the first model layer is the model layer in the first sorting position among the multiple model layers, and the last model layer is the model layer in the last sorting position among the multiple model layers.

[0036] Among them, the random confusion low-rank matrix L′ can be determined based on the true random number generator in the Trusted Execution Environment (TEE) to enhance security and reduce the risk of privacy data (x, y) being restored. Specifically, in the embodiment of the present disclosure, in the scenario of fine-tuning based on the general pre-trained large model, the dense domain model Model Llama2-7B-Crypt It is known or public, and its output will be transmitted to an incompletely trusted plaintext computing environment. When both the model and the output are known, the original private data can be restored to a certain extent through the inversion method. Based on this, it is necessary to initialize a random confusion low-rank matrix L′ inside the TEE during the initialization phase and connect it in parallel to the first model layer to protect common private data. Among them, in the disclosed embodiment, the random confusion low-rank matrix can be generated by the true random number generator of the TEE, and the random confusion low-rank matrix can reduce computing power consumption and avoid affecting the model training performance.

[0037] Among them, the first model layer is connected in parallel with the initial low-rank matrix, and in the other model layers in the initial large model except the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with the target low-rank matrix.

[0038] Among them, the target low-rank matrix is ​​a low-rank matrix that assists in fine-tuning the initial large model. That is, by updating the parameters in the target low-rank matrix, the large model can be fine-tuned. Since the rank of the low-rank matrix is ​​much smaller than its number of rows and columns, it means that the information in the matrix can be represented by fewer parameters. In model fine-tuning, by introducing the target low-rank matrix, the number of parameters that need to be trained can be significantly reduced, and due to the significant reduction in the number of parameters, the computational efficiency of the model will be significantly improved. During the fine-tuning and training process, this can reduce the consumption of computing resources and speed up the fine-tuning training.

[0039] S102: The first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as a secret domain model, and the secret domain model is deployed in the trusted execution environment TEE.

[0040] In the embodiments of the present disclosure, see Figure 2 , Figure 2 is a schematic diagram of the training process of the split large model proposed in an embodiment of the present disclosure, Figure 2 Take the Transformers large model with Decoder Only structure as an example. Assume that the large model has L 1 To L n There are n layers of model (transformer) blocks. During the large model inference process, only the initial L 1The loss function of the Ln layer and the final Ln layer are identified as the dense domain model. In actual business scenarios, the target low-rank matrix (such as Figure 2 The parameters in the Lora in are adjusted, so the target low-rank matrix needs to be split down as a secret domain model to protect it in a trusted execution environment. For subsequent convenience, the split secret domain models are collectively referred to as Model Llama2-7B-Crypt .

[0041] Among them, Trusted Execution Environment (TEE) is an independent processing environment with computing and storage functions that can provide security, integrity and confidentiality protection. It can realize confidential computing capabilities such as data confidentiality out of the domain, non-recoverable non-trusted environment, and controllable out-of-domain data processing logic. The present disclosure is applicable to various hardware implementations of TEE, not limited to specific hardware.

[0042] S103: Use the other model layers except the first model layer and the last model layer in the initial large model as plaintext domain models, and deploy the plaintext domain models in a plaintext computing environment.

[0043] Among them, see above Figure 2 , which can be the other model layers (i.e. L 2 To L n-1 layer) as the plaintext domain model.

[0044] Among them, the plaintext computing environment includes but is not limited to the graphics processing unit (GPU), tensor processing unit (TPU), and neural network processor (NPU) environment.

[0045] That is to say, in the embodiment of the present disclosure, it is necessary to physically split the plaintext domain model and the secret domain model identified above, so as to subsequently add a privacy protection device to the secret domain model. Of course, the various model layers after the split are still treated as a whole in the inference, and the split should not change the key structure and integrity of the original initial large model, nor will it change the output of the initial large model.

[0046] S104: According to the output of each model layer in the dense domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain the target large model.

[0047] In the disclosed embodiment, after the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as secret domain models, and the secret domain model is deployed in a trusted execution environment TEE, and the other model layers except the first model layer and the last model layer in the initial large model are used as plaintext domain models, and the plaintext domain model is deployed in a plaintext computing environment, the parameters of the target low-rank matrix can be updated according to the output of the secret domain model and each model layer in the plaintext domain model to obtain the target large model.

[0048] Optionally, in some embodiments, the parameters of the target low-rank matrix are updated according to the output of each model layer in the dense domain model and the plaintext domain model to obtain the target large model. During the forward propagation process, the target output data of the previous model layer is continuously obtained, and the target output data is used as the target input data of the next model layer until the predicted data of the last model layer output is obtained, and the target loss value of the initial large model is determined according to the predicted data. During the backward propagation process, the parameters of the target low-rank matrix are updated according to the target loss value to obtain the target large model.

[0049] In the embodiments of the present disclosure, see the above Figure 2 , that is, each iteration of model training includes forward propagation and backward propagation. During forward propagation, the output x of the front model 1 =L 1 (x) is used as the input of the backward model, and is sent by remote communication or cross-process communication. 1 ) is sent to the backend model, and the backend model Recv(x 1 ) After receiving the front-end input, it continues to propagate backward and generates output x 2 =L 2 (x 1 ). Backward Propagation Principle

[0050] After the complete split, the training path of the large model is formed.

[0051] Optionally, in some embodiments, obtaining the output data of the previous model layer may be, when the model layer is connected in parallel with the target low-rank matrix, obtaining first output data of the model layer and second output data of the target low-rank matrix, and using the sum of the first output data and the second output data as target input data of the next model layer of the model layer connected in parallel with the target low-rank matrix.

[0052] In the embodiments of the present disclosure, see Figure 3 , Figure 3It is a local flow diagram of model fine-tuning proposed in one embodiment of the present disclosure, that is, the training process after Lora splitting is similar to the longitudinal model segmentation, and is also divided into two processes: forward propagation and back propagation. During forward propagation, the input x is remotely transmitted to the dense domain model (target low-rank matrix) through the Send(x) method, and the output b=B(x) is obtained after being processed by the dense domain model (target low-rank matrix), and then transmitted back to the plaintext domain model through Send(b), and then the outputs of the plaintext domain and the dense domain model (target low-rank matrix) are superimposed y=a+b as the total output of the parallel matrix and propagated backward. Backward propagation is similar, the gradient data is shunted to the dense domain model, and the forward gradient is calculated and then transmitted back to the plaintext domain model for gradient merging.

[0053] That is to say, in the embodiments of the present disclosure, it is necessary to update the parameters of the target low-rank matrix by combining the outputs of each model layer in the dense domain model and the plaintext domain model during the forward propagation and backward propagation of the model, so as to realize the fine-tuning process of the initial large model and obtain the target large model.

[0054] In the disclosed embodiment, an initial large model is obtained, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and in the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix, and then the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as secret domain models, and the secret domain model is deployed in a trusted execution environment TEE, and the model layers other than the first model layer and the last model layer of the initial large model are used as plaintext domain models, and the plaintext domain model is deployed in a plaintext computing environment, and according to the output of each model layer in the secret domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain the target large model, thereby ensuring that the model is not stolen during the fine-tuning of the large model, thereby achieving protection of the model, and since the large model is fine-tuned in conjunction with the target low-rank matrix, the efficiency of model fine-tuning training can also be improved.

[0055] Figure 4 It is a flowchart of a large model fine-tuning method proposed in an embodiment of the present disclosure.

[0056] Among them, it should be noted that the executor of the large model fine-tuning method of this embodiment is a large model fine-tuning device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, and the electronic device can include but is not limited to a terminal, a server, etc. For example, the terminal can be a mobile phone, a handheld computer, etc.

[0057] like Figure 4As shown, the large model fine-tuning method includes:

[0058] S401: Obtain an initial large model, wherein the initial large model includes: a plurality of model layers connected sequentially, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix.

[0059] S402: The first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as the secret domain model, and the secret domain model is deployed in the trusted execution environment TEE.

[0060] S403: Use the other model layers except the first model layer and the last model layer in the initial large model as plaintext domain models, and deploy the plaintext domain models in a plaintext computing environment.

[0061] The specific description of S401 - S403 can be found in the above embodiments, which will not be repeated here.

[0062] S404: Based on the asymmetric root key pair of the cryptographic algorithm of TEE, code measurement is performed on the secret domain model to derive the private key ModelSK and the public key ModelPK of the secret domain model.

[0063] Optionally, in some embodiments, based on the asymmetric root key pair of the cryptographic algorithm of the TEE, code measurement is performed on the secret domain model to derive the private key ModelSK and public key ModelPK of the secret domain model. The interface of the trusted execution environment code measurement may be called to pass in the required secret domain model; based on the passed in secret domain model, a hash value of the secret domain model is calculated using a cryptographic algorithm, and the hash value is used as the fingerprint of the secret domain model; the root private key, random number, and fingerprint that are not visible to the outside world in the asymmetric root key pair of the cryptographic algorithm are used to derive the private key ModelSK of the secret domain model through a cryptographic algorithm; based on the private key ModelSK of the secret domain model, the public key ModelPK of the secret domain model is generated, thereby facilitating the subsequent addition of a privacy protection device to the secret domain model.

[0064] In some embodiments, after generating the public key ModelPK of the secret domain model based on the private key ModelSK of the secret domain model, it also includes: using the root private key of the cryptographic algorithm built into the trusted execution environment to perform a credibility signature certification on the public key ModelPK to obtain the certification signature of the public key ModelPK, wherein the content of the certification signature includes the fingerprint, the public key ModelPK and the client public key UserPK.

[0065] The cryptographic algorithm may include but is not limited to SM2 / SM3, and this embodiment does not specifically limit this.

[0066] In the embodiments of the present disclosure, see Figure 5 , Figure 5 is a schematic diagram of a trusted execution environment provided by an embodiment of the present disclosure, based on the aforementioned secret domain model Model Llama2-7B-Crypt For example, Figure 5 As shown in the figure, prepare the TEE trusted execution environment. The TEE is pre-set with SM2's asymmetric root key pair RootSK (RSK) and RootPK (RPK). RootSK is the root private key that is invisible to the outside world and is the trusted root of the entire TEE. The root public key RootPK can be made public. At the same time, TEE has built-in support for various cryptographic algorithms SM2 / SM3 of the national secret.

[0067] In the embodiment of the present disclosure, the dense domain model Model Llama2-7B-Crypt Perform code measurement and derive the Model in TEE Llama2-7B-Crypt The private key ModelSK and the public key ModelPK are as follows:

[0068] Call the TEE code measurement interface and pass in the required parameters: the model to be measured (Model Llama2-7B-Crypt ), the client public key UserPK; TEE hardware internally uses the passed-in secret domain model Model Llama2-7B-Crypt Use the SM3 algorithm to calculate the hash value ModelHash, that is, ModelHash = SM3 (Model Llama2-7B-Crypt ), ModelHash can be used as Model Llama2-7B-Crypt fingerprints;

[0069] The root private key ModelSK of the secret domain model is derived through the root private key RootSK of TEE, the random number Nonce, and the fingerprint ModelHash through the SM3 algorithm, that is, ModelSK = SM3 (ModelHash, RSK, Nonce). The root private key ModelSK of the secret domain model can be used to decrypt the ciphertext of the private data. The root private key ModelSK of the secret domain model is generated in the TEE hardware and cannot be exported by any method. Therefore, the root private key ModelSK of the secret domain model cannot be obtained from outside the hardware.

[0070] Through the above Model Llama2-7B-Crypt The root private key ModelSK generates the public key ModelPK, that is, ModelPK = SM2_GetPK (ModelSK). The public key ModelPK can be the key used to encrypt the private data of the client;

[0071] The above public key ModelPK is signed by the built-in root private key of TEE. The content of the signature may include the fingerprint ModelHash, the public key ModelPK and the client public key UserPK, that is, Signature = SM2_Sign (ModelHash | ModelPK | UserPK, RSK), and the signature Signature (Sign) is obtained.

[0072] S405: Push the target private data to TEE, and decrypt the target private data using the private key ModelSK to obtain the private data, wherein the target private data is obtained by encrypting the private data using the public key ModelPK by the client.

[0073] The client uses the key ModelPK generated by the TEE model measurement to encrypt private data, [x], [y] = Encrypt (x, y, ModelPK), and the server collects the encrypted private data of multiple clients and stores it in the server's secret storage as training sample data. And before the client encrypts the data, a very important prerequisite is that it needs to prove ModelPK: confirm that ModelPK is indeed the correct key generated by the correct code measurement of the specified TEE environment, that is, to prove that the TEE environment (RootPK) and the calculation model (ModelPK) generated by ModelPK are correct. Llama2-7B-Crypt ), the output of the model result (UserPK), and whether the three key information match. That is, the generated Signature is verified, Verify_Sign(Signature,Model Llama2-7B-Crypt ,UserPK,RootPK)==True, this step can also be assisted by remote proof provided by the hardware manufacturer of TEE, there is no restriction on this.

[0074] S406: Train the secret domain model and the plaintext domain model according to the privacy data, and obtain the output of each model layer in the secret domain model and the plaintext domain model during the training process.

[0075] In the disclosed embodiment, at the beginning of training, similar to plaintext training, the server uses sampling and other methods to build a confidential batch data set from confidential storage, pushes it to the TEE trusted execution environment, and has kept the decryption key ModelSK of the encryption key ModelPK in the secure isolated storage in the TEE in the code measurement action. At this time, ModelSK is used to decrypt the batch of corpus ciphertext in the TEE into plaintext x, y = Decrypt([x], [y], ModelSK), and then x, y are sent to the confidential domain model for training and fine-tuning of the large model.

[0076] S407: According to the output of each model layer in the dense domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain the target large model.

[0077] The specific description of S407 can be found in the above embodiment, which will not be repeated here.

[0078] S408: Encrypt the secret domain model in the target large model based on the determined client public key UserPK, and send the encrypted target large model to the client.

[0079] In the disclosed embodiment, after updating the parameters of the target low-rank matrix according to the output of each model layer in the secret domain model and the plaintext domain model to obtain the target large model, it can be determined that the target large model meets the index requirements. At this time, the secret domain model in the target large model will be encrypted based on the determined client public key UserPK, and the encrypted target large model will be sent to the client. At this time, the model can only be decrypted by the output party, thereby achieving protection of the large model and ensuring that the high-value large model will not be leaked during the training process using outsourced computing power.

[0080] In the disclosed embodiment, an initial large model is obtained, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and in the other model layers except the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix, and then the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as a secret domain model, and the secret domain model is deployed in a trusted execution environment TEE, and then the other model layers except the first model layer and the last model layer in the initial large model are used as plaintext domain models, and the plaintext domain model is deployed in a plaintext computing environment, and then based on the asymmetric root key pair of the cryptographic algorithm of the TEE, code measurement is performed on the secret domain model to derive the private key ModelSK and the public key Mode of the secret domain model. lPK, and then push the target privacy data to TEE, and decrypt the target privacy data through the private key ModelSK to obtain the privacy data, wherein the target privacy data is obtained by the client using the public key ModelPK to encrypt the privacy data, and then train the secret domain model and the plaintext domain model according to the privacy data, and obtain the output of each model layer in the secret domain model and the plaintext domain model during the training process, and then update the parameters of the target low-rank matrix according to the output of each model layer in the secret domain model and the plaintext domain model to obtain the target large model, and then encrypt the secret domain model in the target large model based on the determined client public key UserPK, and send the encrypted target large model to the client, thereby ensuring that the large model can only be decrypted by the output party, thereby achieving protection of the large model and ensuring that high-value large models will not be leaked during training using outsourced computing power.

[0081] Figure 6 It is a schematic diagram of the structure of a large model fine-tuning device proposed in one embodiment of the present disclosure.

[0082] like Figure 6 As shown, the large model fine-tuning device 60 comprises:

[0083] An acquisition module 601 is used to acquire an initial large model, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix;

[0084] The first deployment module 602 is used to use the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer as a secret domain model, and deploy the secret domain model in the trusted execution environment TEE;

[0085] The second deployment module 603 is used to use the model layers except the first model layer and the last model layer in the initial large model as plaintext domain models, and deploy the plaintext domain models in the plaintext computing environment;

[0086] The updating module 604 is used to update the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain the target large model.

[0087] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0088] Before updating the parameters of the target low-rank matrix according to the output of each model layer in the secret domain model and the plaintext domain model to obtain the target large model, the secret domain model is coded and measured based on the asymmetric root key pair of the cryptographic algorithm of TEE to derive the private key ModelSK and public key ModelPK of the secret domain model.

[0089] Push the target private data to TEE, and decrypt the target private data using the private key ModelSK to obtain the private data, where the target private data is encrypted by the client using the public key ModelPK;

[0090] The secret domain model and the plaintext domain model are trained according to the privacy data, and the output of each model layer in the secret domain model and the plaintext domain model is obtained during the training process.

[0091] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0092] Call the trusted execution environment code measurement interface and pass in the required secret domain model;

[0093] According to the input secret domain model, the hash value of the secret domain model is calculated using a cryptographic algorithm, and the hash value is used as the fingerprint of the secret domain model;

[0094] The private key ModelSK of the secret domain model is derived through the cryptographic algorithm using the invisible root private key, random number, and fingerprint in the asymmetric root key pair of the cryptographic algorithm;

[0095] Based on the private key ModelSK of the secret domain model, the public key ModelPK of the secret domain model is generated.

[0096] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0097] During the forward propagation process, the target output data of the previous model layer is continuously obtained, and the target output data is used as the target input data of the next model layer until the predicted data output by the last model layer is obtained;

[0098] Determine the target loss value of the initial large model based on the predicted data;

[0099] During the back-propagation process, the parameters of the target low-rank matrix are updated according to the target loss value to obtain the target large model.

[0100] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0101] When the model layer is connected in parallel with the target low-rank matrix, first output data output by the model layer and second output data output by the target low-rank matrix are obtained;

[0102] The sum of the first output data and the second output data is used as the target input data of the next model layer of the model layer connected in parallel with the target low-rank matrix.

[0103] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0104] In the process of back propagation, the first gradient data corresponding to the next model layer of the model layer connected in parallel with the target low-rank matrix is ​​determined according to the target loss value;

[0105] According to the first gradient data, the parameters of the target low-rank matrix are updated to obtain a target large model.

[0106] In some embodiments of the present disclosure, the update module 604 is further configured to:

[0107] After updating the parameters of the target low-rank matrix according to the output of each model layer in the secret domain model and the plaintext domain model to obtain the target large model, the secret domain model in the target large model is encrypted based on the determined client public key UserPK, and the encrypted target large model is sent to the client.

[0108] It should be noted that the aforementioned explanation of the large model fine-tuning method is also applicable to the large model fine-tuning device of this embodiment and will not be repeated here.

[0109] In the disclosed embodiment, an initial large model is obtained, wherein the initial large model includes: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and in the model layers other than the first model layer and the last model layer of the initial large model, at least one model layer is connected in parallel with a target low-rank matrix, and then the first model layer of the initial large model and its parallel initial low-rank matrix, the target low-rank matrix and the last model layer are used as secret domain models, and the secret domain model is deployed in a trusted execution environment TEE, and the model layers other than the first model layer and the last model layer of the initial large model are used as plaintext domain models, and the plaintext domain model is deployed in a plaintext computing environment, and according to the output of each model layer in the secret domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain the target large model, thereby ensuring that the model is not stolen during the fine-tuning of the large model, thereby achieving protection of the model, and since the large model is fine-tuned in conjunction with the target low-rank matrix, the efficiency of model fine-tuning training can also be improved.

[0110] Figure 7 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. Figure 7 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0111] like Figure 7 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0112] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0113] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0114] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 Not shown, often called a "hard drive").

[0115] although Figure 7 Not shown, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (Compact Disc Read Only Memory; hereinafter referred to as: CD-ROM), a digital versatile disc read only memory (Digital Video Disc Read Only Memory; hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0116] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0117] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable the human body to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0118] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the large model fine-tuning method mentioned in the above embodiment.

[0119] In order to implement the above embodiments, the present disclosure also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the large model fine-tuning method proposed in the above embodiments of the present disclosure is implemented.

[0120] In order to implement the above embodiments, the present disclosure further proposes a computer program product. When an instruction processor in the computer program product executes, the large model fine-tuning method proposed in the above embodiments of the present disclosure is executed.

[0121] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0122] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0123] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0124] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0125] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0126] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0127] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0128] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0129] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0130] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A large model fine-tuning method, characterized in that: The method comprises: Acquire an initial large model, wherein the initial large model comprises: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the other model layers in the initial large model except the first model layer and the last model layer of the initial large model, at least one of the model layers is connected in parallel with a target low-rank matrix; The first model layer of the initial large model and the initial low-rank matrix in parallel, the target low-rank matrix and the last model layer are used as a secret domain model, and the secret domain model is deployed in a trusted execution environment TEE; Using the other model layers in the initial large model except the first model layer and the last model layer as plaintext domain models, and deploying the plaintext domain models in a plaintext computing environment; According to the output of each model layer in the dense domain model and the plaintext domain model, the parameters of the target low-rank matrix are updated to obtain a target large model.

2. The method according to claim 1, characterized in that Before updating the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain the target large model, the method further includes: Based on the asymmetric root key pair of the cryptographic algorithm of the TEE, code measurement is performed on the secret domain model to derive the private key ModelSK and the public key ModelPK of the secret domain model; Push the target private data to the TEE, and decrypt the target private data using the private key ModelSK to obtain the private data, wherein the target private data is obtained by encrypting the private data using the public key ModelPK by the client; The secret domain model and the plaintext domain model are trained according to the privacy data, and the output of each model layer in the secret domain model and the plaintext domain model is obtained during the training process.

3. The method according to claim 2, characterized in that The asymmetric root key pair based on the cryptographic algorithm of the TEE performs code measurement on the secret domain model to derive the private key ModelSK and the public key ModelPK of the secret domain model, including: Call the trusted execution environment code measurement interface and pass in the required secret domain model; According to the input secret domain model, a hash value of the secret domain model is calculated using a cryptographic algorithm, and the hash value is used as the fingerprint of the secret domain model; Using the root private key, a random number and the fingerprint in the asymmetric root key pair of the cryptographic algorithm, which is invisible to the outside, to derive the private key ModelSK of the secret domain model through the cryptographic algorithm; Based on the private key ModelSK of the secret domain model, the public key ModelPK of the secret domain model is generated.

4. The method according to claim 1, characterized in that The updating of the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain a target large model includes: During the forward propagation process, the target output data of the previous model layer is continuously obtained, and the target output data is used as the target input data of the next model layer, until the predicted data output by the last model layer is obtained; Determine the target loss value of the initial large model according to the predicted data; During the back propagation process, the parameters of the target low-rank matrix are updated according to the target loss value to obtain the target large model.

5. The method according to claim 4, characterized in that The step of obtaining target output data of a previous model layer and using the target output data as target input data of a subsequent model layer includes: In the case where the model layer is connected in parallel with the target low-rank matrix, obtaining first output data output by the model layer and second output data output by the target low-rank matrix; The sum of the first output data and the second output data is used as target input data of a subsequent model layer of the model layer connected in parallel with the target low-rank matrix.

6. The method according to claim 5, characterized in that In the back propagation process, the parameters of the target low-rank matrix are updated according to the target loss value to obtain the target large model, including: In the process of back propagation, determining first gradient data corresponding to a subsequent model layer of the model layer connected in parallel with the target low-rank matrix according to the target loss value; According to the first gradient data, the parameters of the target low-rank matrix are updated to obtain the target large model.

7. The method according to any one of claims 1 to 6, characterized in that After updating the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain the target large model, the method further includes: The secret domain model in the target large model is encrypted based on the determined client public key UserPK, and the encrypted target large model is sent to the client.

8. A large model fine-tuning device, characterized in that: The device comprises: An acquisition module is used to acquire an initial large model, wherein the initial large model comprises: a plurality of model layers connected in sequence, wherein the first model layer is connected in parallel with an initial low-rank matrix, and among the other model layers in the initial large model except the first model layer and the last model layer of the initial large model, at least one of the model layers is connected in parallel with a target low-rank matrix; A first deployment module is used to use the first model layer of the initial large model and the initial low-rank matrix in parallel, the target low-rank matrix and the last model layer as a secret domain model, and deploy the secret domain model in a trusted execution environment TEE; A second deployment module is used to use the other model layers in the initial large model except the first model layer and the last model layer as plaintext domain models, and deploy the plaintext domain models in a plaintext computing environment; An updating module is used to update the parameters of the target low-rank matrix according to the output of each model layer in the dense domain model and the plaintext domain model to obtain a target large model.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images