Large language model parameter protection method and device based on permutation and zooming technology

Abstract

A large language model parameter protection method and device based on a replacement and scaling technology are mainly divided into an initialization stage and a model reasoning stage: in the initialization stage, replacement and scaling parameters of model conversion are generated, and the model parameters are converted according to the replacement and scaling values; and in the model reasoning stage, the input is subjected to replacement conversion, reasoning operation is performed according to the model parameters generated in the initialization stage on the basis of the converted input in a normal transform calculation mode to obtain a calculation result, and then the calculation result is subjected to replacement reduction to obtain real output. The method is completely compatible with an attention mechanism transformer structure containing ROPE, the adaptability with a newest model architecture is ensured, and the accuracy of consistency with an original model and almost lossless high efficiency can be ensured while model parameters are protected; the method has a wide application prospect in the fields of large language model security deployment, privacy protection calculation, cross-mechanism AI collaboration and the like.

Description

Technical Field

[0001] The present invention relates to the fields of security, artificial intelligence, cloud computing, big data and edge computing technology, and in particular to a large language model parameter protection method and device based on substitution and scaling technology. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) are increasingly used in various fields. As an important data asset, the security protection of large language models has become a key issue. At present, common protection schemes are generally based on technologies such as homomorphic encryption, multi-party secure computing, differential privacy, and trusted execution environment. However, these schemes have significant limitations:

[0003] 1. Although homomorphic encryption and multi-party secure computing provide strong security guarantees, the computational overhead is huge, which seriously affects the reasoning efficiency of the model;

[0004] 2. Differential privacy is mainly aimed at protecting training / prediction data, but has limited security protection for the model itself and will lose a certain degree of accuracy;

[0005] 3. The trusted execution environment relies on specific hardware, has a limited deployment scope, and may be at risk of side-channel attacks.

[0006] In view of these limitations, researchers proposed the permutation-based STIP (Secure Transformer Inference Protocol). The core advantages of the STIP protocol are:

[0007] 1. Maintain model accuracy: Ensure that model performance is not affected through clever permutation operations;

[0008] 2. Efficient reasoning performance: Compared with encryption schemes, STIP has a lower computational overhead, ensuring reasoning efficiency;

[0009] 3. Model / data privacy protection: Utilizing the high-dimensional characteristics of word vector length (usually 512 to 4098) and model parameters, brute force cracking becomes extremely difficult.

[0010] The basic principle of the STIP protocol is to use permutation to protect the privacy of model parameters and user data without affecting the accuracy of the model. This is because the permutation operation does not essentially change the statistical properties of the data, but only rearranges the order of the data.

[0011] However, the STIP protocol encountered challenges when facing Rotary Position Embedding (ROPE), which is widely used in the Transformer architecture of large language models. The main problem is that ROPE needs to operate on cosine (cos) and sine (sin) vectors, which are directly related to the sentence length and can be regarded as constants. If these vectors are permuted, it is easy to know the permutation method through comparison, which makes the protection of query (Q) and key (K) parameters invalid. This limitation seriously restricts the application of the STIP protocol in large language models containing ROPE structures.

[0012] Related term explanation:

[0013] Large Language Model (LLM): A large-scale natural language processing model trained based on deep learning technology. These models usually contain billions to hundreds of billions of parameters, can understand and generate human language, and perform various complex natural language processing tasks such as text generation, question answering, translation, etc.

[0014] Inference: refers to the process of using the trained model to process and predict new input data. In large language models, inference may include generating text, answering questions, completing tasks, etc. The inference process is usually faster than the training process, but it still requires a lot of computing resources, especially for large-scale models.

[0015] Rotary Position Embedding (ROPE): is an advanced position encoding technique for Transformer models. It encodes the relative position information of words in a sequence by applying a rotation transformation to word embeddings. ROPE can effectively capture positional dependencies in a sequence while maintaining the extrapolation ability of the model. This encoding method performs well in processing variable-length sequences and has been widely used in the latest large language model architectures. The implementation of ROPE typically involves applying cosine and sine functions to the query (Q) and key (K) vectors, which are directly related to the position of the input sequence. Summary of the invention

[0016] The present disclosure provides a large language model parameter protection method and device based on permutation and scaling technology, which can not only maintain the advantages of the STIP protocol (consistent accuracy and high efficiency), but also effectively support emerging Transformer structures such as ROPE. The method works in the following two scenarios:

[0017] Large models are deployed in the user environment: This prevents attackers from stealing large model parameters while fully controlling the user environment;

[0018] Large models are deployed in cloud service provider environments: This prevents cloud service providers from stealing model parameters and provides data protection solutions for users.

[0019] The large language model parameter protection method based on the permutation and scaling technology provided in the present disclosure mainly includes the following steps:

[0020] Initialization phase:

[0021] S1, permutation and scaling parameters for generative model transformation;

[0022] S2, transforming the model parameters according to the displacement and scaling values ​​obtained in step S1;

[0023] Model inference phase:

[0024] S3, performs permutation transformation on the input;

[0025] S4, in accordance with the normal transformer calculation method, based on the transformed input and the model parameters generated in the initialization phase, the calculation results are obtained by inference operation;

[0026] S5, permuting and restoring the calculation result to obtain the real output.

[0027] This paper integrates permutation and scaling techniques to construct a Transformer parameter conversion method suitable for ROPE: by cleverly using scaling technology, parameter protection of Q and K is achieved while maintaining compatibility with ROPE, thereby providing a more flexible and efficient parameter conversion solution; it solves the limitations of pure soft implementations (such as the stip protocol) in processing the attention mechanism of ROPE after Q and K, which is widely used in modern large-scale language models.

[0028] The Transformer parameter protection system based on permutation and scaling technology using the above method is a comprehensive parameter protection solution, which mainly includes the following key modules:

[0029] 1) Model parameter conversion module: used in the initialization phase to ensure the security of model parameters;

[0030] 2) Input conversion module: processes input data during the inference phase;

[0031] 3) Inference computing module: performs core inference operations while protecting the privacy of the model;

[0032] 4) Output restoration module: ensure the security and correctness of the output results.

[0033] This system architecture provides end-to-end parameter protection for the Transformer model.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] (1) The permutation matrix of QK is replaced by the scaling matrix, which can be applied to the Transformer parameter conversion method of ROPE, thus expanding the scope of application;

[0036] (2) At the same time, it retains the advantages of the permutation-based STIP protocol technology - the same accuracy as the original model and almost lossless efficiency;

[0037] (3) A system device for protecting the circulation of large models in a two-party scenario in a purely soft environment is presented. Its significant advantages are: it supports the transformer structure with the attention mechanism of ROPE, which expands the scope of application; it adopts pure software implementation, gets rid of the dependence on hardware such as the trusted execution environment (TEE), and improves the flexibility and cost-effectiveness of deployment; thus providing strong protection for the secure transaction of large language models;

[0038] (4) A large language model security protection method and device in a three-party scenario are proposed. The method and device are fully compatible with the attention mechanism transformer structure containing ROPE, ensuring compatibility with the latest model architecture. At the same time, the model security is maintained in complex three-party interactions, providing a comprehensive privacy protection solution for cloud-based large language model services. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0040] Figure 1 A schematic diagram of the structure of a large language model with ROPE in the attention mechanism;

[0041] Figure 2 is a parameter conversion method according to an exemplary embodiment of the present disclosure;

[0042] Figure 3 It is a schematic diagram of the use of a large model for parameter protection in a two-party scenario;

[0043] Figure 4 This is a schematic diagram of the large model for parameter protection in a three-party scenario. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] The present disclosure provides a large language model parameter protection method and device based on permutation and scaling technology, which can effectively support the Transformer structure including rotation position encoding (ROPE). It mainly solves the following problems:

[0046] (1) Extend the stip scheme to support Transformer models that include emerging structures such as ROPE;

[0047] (2) Maintaining the accuracy of large models: While providing security protection for large language models, the reasoning accuracy of large language models is not lost;

[0048] (3) Efficient possible reasoning performance: ensuring the security of large language models while ensuring that the reasoning speed is hardly reduced;

[0049] (4) Provide two methods for secure reasoning of large models in two-party / three-party mode.

[0050] The large language model parameter protection solution based on permutation and scaling technology provided by the present disclosure is mainly divided into two stages: initialization stage and model inference stage:

[0051] In the initialization phase, the model parameter conversion module is responsible for generating the displacement and scaling parameters of the model conversion, and converting the model parameters according to these displacement and scaling values;

[0052] In the model inference stage, the input conversion module is responsible for permuting the input. The model calculation module performs inference operations based on the converted input and the model parameters generated in the initialization stage in the normal transformer calculation method to obtain the calculation results. The output restoration model is responsible for permuting and restoring the calculation results to obtain the real output.

[0053] In an exemplary embodiment, as shown in the attached Figure 1 The figure shows a schematic diagram of the structure of a large language model with an attention mechanism containing ROPE, where input -> embedding is the part calculated by the model user; the model parameter part includes Layer1, Layer2, ..., LayerN and the normalization layer. These parameters need to be converted and maintained and calculated by the model server; the final linear layer and Softmax layer are also calculated by the model user.

[0054] The specific process of protecting the model parameters is as follows:

[0055] 1. Initialization phase: start the model parameter conversion module

[0056] Step 1. Selection of displacement parameters and scaling parameters:

[0057] Random permutation π∈{0,1} d×d , where d is the feature dimension; for the i-th layer transformer, where i∈{1,2,…,N}, two permutation matrices π are randomly selected i,1 ,π i,2 ∈{0,1} d×d and a random scaling matrix μ i The form of a diagonal matrix is ​​as follows:

[0058] For the i-th layer transformer, where i∈{1,2,…,N}, randomly select two permutation matrices π i,1 ,π i,2 ∈{0,1} d×d and a random scaling matrix μ i The form of a diagonal matrix is ​​as follows:

[0059]

[0060] Among them, s j Corresponding to the scaled value in dimension j, j∈[d];

[0061] Without loss of generality, we assume that a multi-head attention mechanism is used, and the number of attention heads is denoted as h, so each group contains d h =d / / h elements. μ i Further grouping is shown as follows:

[0062]

[0063] S0,S1,…,S h-1 The diagonal matrices of length d_h are shown below, with S j For example, s j,0 ,s j,1 ,…,s j,dh / / 2-1 The values ​​are random values ​​selected between [0.5, 1.5], and the scaling values ​​appear in pairs. This is because it is necessary to ensure that the scaling invariance can be maintained after ROPE.

[0064]

[0065] Step 2: Based on the above permutation and scaling matrix, π∈{0,1} d×d , {πi,1 ,π i,2 ,μ i} i∈[N] Transform the model parameters:

[0066] 2.1 Parameter conversion of multi-head attention mechanism with ROPE

[0067] For i∈{1,2,…,N}, L i Parameter W in the multi-head attention mechanism in the layer q ,W k ,W v ,W o Divided into conversions:

[0068] W q '=π T W q μ i ,W k '=π T W k μ i -1 ,

[0069] W v '=π T W v π i,1 ,W o '=π i,1 T W o π,

[0070] 2.2 Parameter conversion of normalization layer

[0071] An example of an embodiment is shown in FIG. Figure 2 As shown, RMSNorm is used for two normalizations, and the corresponding parameters γ1 and γ2 are transformed into:

[0072] γ1'=γ1π,γ2'=γ2π

[0073] Another possible implementation uses LayerNorm to perform normalization twice, and the corresponding parameters γ1, β1, γ2, and β2 are converted into:

[0074] γ1'=γ1π, β1'=β1π, γ2'=γ2π, β2'=β2π

[0075] 2.3 The parameters in MLP are transformed into:

[0076] An example of an embodiment is shown in FIG. Figure 2 As shown, using FFN (SWiGLU), this can be expressed as

[0077]

[0078] Further up ,W gate ,W down The parameters are converted into:

[0079]

[0080] Another embodiment uses FFN (ReLU), expressed as

[0081] FFN RelU (x) = RELU(xW1)W2

[0082] The W1 and W2 parameters are transformed into:

[0083] W1'=π T W1π i,2 ,W2'=π i,2 T W2π

[0084] 2.4 For the normalized layer parameters after LayerN, perform parameter conversion in the same way as in step 2.2.

[0085] It can be seen that after the above model parameter conversion (in W q '=π T W q μ i For example), even if the attacker can obtain F θ′ , and master the permutation π, it is also impossible to know the original parameters (i.e. W q ).

[0086] 2. Model inference stage: Model calculation method for parameter protection:

[0087] Step 1, input conversion module:

[0088] According to the permutation π∈{0,1} d×d For input Transform to obtain a new input x'=xπ, where n is the sequence length and d is the feature dimension;

[0089] Step 2, model reasoning calculation module:

[0090] The original model parameters are denoted as θ, and the transformed parameters are denoted as θ', F θ is the transfomer model with original parameters, F θ' For the transformer model with transformed parameters, the model parameter inference calculation module is calculated in the same way as the normal transformer.

[0091] In order to show the calculation results, the following is a detailed description of Fθ' (xπ)=F θ (x)π:

[0092] 2.1 After the attention mechanism parameter conversion, the multi-head attention mechanism maintains permutation invariance, that is, u'=Att'(xπ)=Att(x)π=uπ:

[0093] Taking xπ as input, we calculate

[0094] Q'=xπW q '=xππ T W q μ i =Qμ i ,

[0095] K'=xπW k '=xππ T W k μ i -1 =Kμ i -1 ,

[0096] V'=xπW v '=xππ T W v π i,1 =Vπ i,1

[0097] After further ROPE, Q' and K' can be expressed as follows, where m represents position m

[0098]

[0099] Among them, the representation of ROPE of Q can be regarded as f(Q,m)=R d Q, Q=(q0,q1,…,q d-1 )

[0100] In this way, we can get

[0101] ROPE(Q′)=f(Q′,m)=f(Q,m)·μ i =ROPE(Q)·μ i ,

[0102] ROPE(K')=f(K',m)=f(K,m)·μ i -1 =ROPE(K)·μ i -1

[0103]

[0104] 2.2 Normalized parameter transformation maintains permutation invariance

[0105] 1) Through the RMSNorm parameter conversion, RMSNorm maintains permutation invariance:

[0106] 2) LayerNorm conversion, LayerNorm maintains permutation invariance:

[0107] 2.3 Feedforward Networks Maintain Permutation Invariance

[0108] 1) FFN (SWiGLU) maintains permutation invariance:

[0109]

[0110] 2) FFN (ReLU) maintains permutation invariance:

[0111]

[0112] 2.4 For the normalization layer parameters after Layer N, perform parameter conversion in the same way as in step 2.2, so as to ensure that Norm(xπ)=Norm(x)π.

[0113] Therefore, it is not difficult to see that after the above parameter changes, F θ′ (xπ)=F θ (x)π.

[0114] Step 3, output the restoration module:

[0115] The final output of step 2 is recorded as z'=zπ. After that, the model user generally needs to go through the linear layer lm_head and the softmax layer:

[0116] 3.1 First, use the permutation π to restore the original value, that is, calculate z = z'π T = zππ T ;

[0117] 3.2 Normal calculation of lm_head layer: lm-head(z'π T ) = lm-head(z);

[0118] 3.3 Calculate the softmax layer normally and get the final output y = softmax (lm-head (z)) which is consistent with the result of the original model.

[0119] Consistent accuracy: After steps 1-3 above, it can be seen that the output of the converted model is consistent with the original model output;

[0120] Efficiency: Although the input, output, and intermediate parameters have undergone permutation and scaling, this does not increase the computational complexity of the model. There are two reasons for this phenomenon: first, the permutation and scaling operations do not change the dimensional structure of the original parameters; second, these operations can maintain the sparse characteristics of the matrix. Therefore, the main performance loss is only the increase in input / output transmission communication performance in the three-party scenario, the token transmitted by the original model, rather than the high-dimensional embedding.

[0121] For scenarios where models are traded and circulated between two or three parties as important data assets, this disclosure proposes an innovative security protection solution.

[0122] 1. Method and device for protecting large language model parameters in two-party scenarios:

[0123] The two parties in the two-party model refer to the large model developer and the large model user. The large model user purchases the large model developed by the large model developer and deploys and runs it on its own infrastructure. In order to ensure the intellectual property rights of the large model, the large model parameters are protected based on the replacement and scaling technology. The specific method and device in this embodiment are as shown in the attached Figure 3 As shown:

[0124] The model parameter conversion module is deployed in the large model developer, and the input conversion module, model calculation module and output restoration module are all packaged in the computing sandbox and finally run on the infrastructure of the model user.

[0125] a. The model developer owns the original model F θ , call the model conversion module and randomly select the permutation π∈{0,1} according to step 1 of the initialization phase d×d ,{π i,1 ∈{0,1} d×d , Scaling Matrix

[0126] And according to step 2 of the initialization phase, the relevant parameters are permuted or scaled to obtain the converted model F θ’ ;

[0127] b. The model developer will convert the converted model F θ’ and π in the form of a computing sandbox (containing an input conversion module, a model calculation module, and an output restoration module) and sent to the model user through a secure channel;

[0128] c. The model user runs the computing sandbox on his own infrastructure to use the large model for inference calculations:

[0129] a) The user of the model performs input normally;

[0130] b) Start the permuted input conversion module inside the sandbox and convert it into the permuted embedding;

[0131] c) For the replaced embedding, call the model calculation module to calculate and get the output;

[0132] Call the output restoration module to restore the output based on the permutation to get the corresponding output token; if the output is not a terminal symbol or the maximum sentence length is not reached, continue to repeat the bd process.

[0133] 2. Method and device for protecting large language model parameters in three-party scenarios:

[0134] In the three-party model, the three parties are the big model developer, the big model user, and the model server (cloud service provider). The big model developer provides the big model and deploys it in the cloud for use by other model users. In order to ensure the secure deployment of the big model in the cloud and prevent unauthorized model access, the big model parameters are protected based on the replacement and scaling technology. The specific method and device in this embodiment are as shown in the attached figure. Figure 4 As shown:

[0135] The model parameter conversion module is deployed within the model developer, the model calculation module is deployed within the model server / cloud service provider; the input conversion module and the output restoration module are both deployed within the model user.

[0136] 1. The model developer owns the original model F θ , call the model parameter conversion module and randomly select the permutation π∈{0,1} in step 2 d×d ,{π i,1 ∈{0,1} d×d , Scaling Matrix And replace or scale the relevant parameters according to step 2 to obtain the converted model F θ’ and input-output permutation π;

[0137] 2. The model developer converts the parameters of the model F θ’ The input and output permutations π are transmitted to the cloud service provider and the model user through secure channels respectively;

[0138] 3. The model user replaces π based on its own input x and the received input and output, and calls the input conversion module in the local device to calculate xπ;

[0139] 4. The model user securely transmits the replaced embedding to the cloud service provider;

[0140] 5. Cloud service provider's model based on parameter protection F θ’Call the model computing module with input xπ to perform inference service and get output y';

[0141] 6. The cloud service provider securely transmits the output y' to the model user;

[0142] The model user calls the output restoration module to restore the output y = y'π based on the permutation π T , further run the lm_head layer and softmax layer to convert it into token; if the token is not a terminal symbol or the maximum sentence length has not been reached, continue the above 3-7 process until the final output.

[0143] In the above embodiments, not only the technical difficulties of Transformer parameter conversion suitable for ROPE are cleverly solved, but also the accuracy consistent with the original model and almost lossless efficiency are ensured while protecting the model parameters. This innovative solution has broad application prospects in the fields of secure deployment of large language models, privacy-preserving computing, and cross-institutional AI collaboration.

[0144] By integrating permutation and scaling techniques, we have achieved comprehensive protection for the Transformer model with ROPE attention mechanism, building a secure and trusted AI ecosystem for model owners, service providers, and end users.

[0145] In addition, the pure software implementation of this solution greatly reduces the deployment threshold, making it easy to integrate into existing cloud computing platforms and edge computing devices, providing a strong protection mechanism for large models, a core data asset, so that model owners can safely put them into market circulation without worrying about the leakage of core technology.

[0146] This technology can be widely used in financial risk control, medical diagnosis, intelligent manufacturing and other fields that have extremely high requirements for model security and performance and require data flow.

[0147] The above technical scheme is only an exemplary embodiment of the present invention. For those skilled in the art, it is easy to make various types of improvements or modifications based on the application methods and principles disclosed in the present invention, and it is not limited to the method described in the above specific embodiment of the present invention. Therefore, the method described above is only preferred and does not have a restrictive meaning.

Claims

1. A large language model parameter protection method based on substitution and scaling technology, comprising the following steps: Initialization phase: S1, permutation and scaling parameters for generative model transformation; S2, transforming the model parameters according to the displacement and scaling values ​​obtained in step S1; Model inference phase: S3, performs permutation transformation on the input; S4, in accordance with the normal transformer calculation method, based on the transformed input and the model parameters generated in the initialization phase, the calculation results are obtained by inference operation; S5, permuting and restoring the calculation result to obtain the real output.

2. The method according to claim 1, characterized in that The step S1 method comprises: Assume that the large language model parameter part includes: Layer1, Layer2, ..., LayerN and normalization layer; Let the random permutation matrix be π∈{0,1} d×d , where d is the feature dimension; For the i-th layer transformer, where i∈{1,2,…,N}, randomly select two permutation matrices π i,1 ,π i,2 ∈{0,1} d×d and a random scaling matrix μ i , μ i is a diagonal matrix of the following form: Among them, sj corresponds to the scaling value in dimension j, j∈[d]; Assume that the large language model uses a multi-head attention mechanism, and the number of attention heads is recorded as h, so each group contains d h =d / / h elements, / / indicates integer divisibility; μ i Further grouping is shown as follows: S0,S1,…,S h-1 The length is d h The diagonal matrix of is as follows, 3. The method according to claim 2, characterized in that The step S2 method comprises: Based on the permutation and scaling matrix obtained in step S1, π∈{0,1} d×d , {π i,1 ,π i,2 ,μ i } i∈[N] Transform the model parameters: S21, parameter conversion of multi-head attention mechanism with ROPE: For i∈{1,2,…,N}, the parameter W in the multi-head attention mechanism in the Layeri layer q ,W k ,W v ,W o Divided into conversions: W q '=π T W q m i ,W k '=π T W k m i -1 , W v '=π T W v p i,1 ,W o '=π i,1 T W o p; S22, parameter conversion of normalization layer: If RMSNorm is used for two normalizations, the corresponding parameters γ1 and γ2 will be transformed into: γ1'=γ1π,γ2'=γ2π If LayerNorm is used for two normalizations, the corresponding parameters γ1, β1, γ2, and β2 will be transformed into: γ1'=γ1π, β1'=β1π, γ2'=γ2π, β2'=β2π; S23, the parameters in MLP are transformed into: If FFN (SWiGLU) is used, that is: Then the parameter W up ,W gate ,W down Converted to: If FFN (ReLU) is used, that is: FFN RelU (x) = RELU(xW1)W2 Then the parameters W1 and W2 are transformed into: W1'=π T W1π i,2 ,W2'=π i,2 T W2π; S24, performing parameter conversion on the normalized layer parameters after Layer N in the same manner as in step S22.

4. The method according to any one of claims 1 to 3, characterized in that: The model reasoning stage specifically includes: S3, input transformation: according to the permutation π∈{0,1} d×d For input Transform to obtain a new input x'=xπ, where n is the sequence length and d is the feature dimension; S4, model inference calculation: the original model parameters are denoted as θ, the converted parameters are denoted as θ', F θ is the transfomer model with original parameters, F θ' The transformer model is the transformed parameter, which is calculated according to the normal transformer calculation method. θ' (xπ)=F θ (x)π; S5: Output restoration: The final output of step S4 is recorded as z'=zπ, and it also needs to go through the linear layer lm_head and the softmax layer: First, use the permutation π to restore the original value, that is, calculate z = z'π T = zππ T ; Normal calculation of lm_head layer: lm-head(z'π T ) = lm-head(z); The softmax layer is calculated normally, and the final output y = softmax (lm-head (z)) is consistent with the result of the original model.

5. A large language model parameter protection device based on substitution and scaling technology using any of the methods described in claims 1-4, characterized in that: include: Model parameter conversion module: used to generate displacement and scaling parameters of model conversion in the initialization phase, and convert the model parameters according to these displacement and scaling values; Input conversion module, used to perform permutation conversion on input in the model inference phase; The model calculation module is used to perform inference operations in the model inference phase in accordance with the normal transformer calculation method based on the transformed input and the model parameters generated in the initialization phase to obtain the calculation results; The output restoration module is used to replace and restore the calculation results to obtain the real output.

6. The device according to claim 5, characterized in that In a two-party scenario consisting of a large model developer and a large model user, the deployment and operation of the device includes: The big model user obtains the big model developed by the big model developer and deploys and runs it on its own infrastructure; The model parameter conversion module is deployed in the large model developer, and the input conversion module, model calculation module and output restoration module are all packaged in the computing sandbox and finally run on the infrastructure of the model user, specifically including: a. The model developer owns the original model F θ , call the model conversion module and randomly select a permutation π∈{0,1} according to step S1 in the initialization phase d×d ,{π i,1 ∈{0,1} d×d , Scaling Matrix And according to step S2 of the initialization phase, the relevant parameters are permuted or scaled to obtain the converted model F θ’ ; b. The model developer will convert the converted model F θ’ and π in the form of a computing sandbox and sent to the model user through a secure channel; the computing sandbox contains the input conversion module, the model computing module, and the output restoration module; c. The model user runs the computing sandbox on its own infrastructure to use the large model for inference calculations: a) The user of the model performs input normally; b) Start the permuted input conversion module inside the sandbox and convert it into the permuted embedding; c) For the replaced embedding, call the model calculation module to calculate and get the output; d) Call the output restoration module to restore the output based on the permutation to obtain the corresponding output token; if the output is not a terminal symbol or the maximum sentence length is not reached, continue to repeat the bd process.

7. The device according to claim 5, characterized in that In a three-party scenario consisting of a large model developer, a large model user, and a model server, i.e., a cloud service provider, the deployment and operation of the device includes: Large model developers provide large models and deploy them in the cloud for use by other model users. The model parameter conversion module is deployed in the model developer, the model calculation module is deployed in the model server / cloud service provider; the input conversion module and the output restoration module are both deployed in the model user, specifically including: (1) The model developer owns the original model F θ , call the model parameter conversion module, and randomly select a permutation π∈{0,1} according to the step S2 d×d ,{π i,1 ∈{0,1} d×d , Scaling Matrix And replace or scale the relevant parameters according to step S2 to obtain the converted model F θ’ and input-output permutation π; (2) The model developer converts the parameters of the model F θ’ The input and output permutations π are transmitted to the cloud service provider and the model user through secure channels respectively; (3) The model user uses its own input x and the received input and output substitution π, and calls the input conversion module in the local device to calculate xπ; (4) The model user securely transmits the replaced embedding to the cloud service provider; (5) Cloud service provider’s model based on parameter protection F θ’ Call the model calculation module with input xπ to perform reasoning service to obtain output y'; (6) The cloud service provider securely transmits the output y' to the model user; (7) The model user calls the output restoration module and restores the output y=y'π based on y' based on the permutation π T , further run the lm_head layer and softmax layer to convert it into tokens; if the token is not a terminal symbol or the maximum sentence length has not been reached, continue the above (3)-(7) process until the final output.

Images