Risk prevention and control method based on security multi-party computing and hybrid expert model

Abstract

The invention discloses a risk prevention and control method based on secure multi-party calculation and a hybrid expert model. The method comprises the following steps: giving an addition secret share, a key sk and a feedforward network weight matrix set; the client C and the server S complete secret state parameter selection and compression by calculating the security product and accumulation of the homomorphic ciphertext of the weight matrix set and the selection vector; and the client C and the server S calculate Kexp experts through the secret state parameters obtained in the selection stage and output the Kexp experts. According to the method, by executing the selection stage, redundant calculation of at least Nexp-Kexp unselected experts is eliminated, sparsity is kept under MoE setting, the extra cost of ciphertext multiplication caused by sparsity is fixed and relatively small, and the optimization effect becomes more and more remarkable along with increasing of the number of the experts.

Description

Technical Field

[0001] This invention belongs to the technical field of secure multi-party computation and hybrid expert models, specifically relating to a risk prevention and control method based on secure multi-party computation and hybrid expert models. Background Technology

[0002] In recent years, Transformer models have greatly enhanced the capabilities of machine learning across a range of tasks. In terms of model scaling, the Hybrid Expert (MoE) architecture has become a powerful technique among Transformer-based models. MoE significantly reduces computational overhead while maintaining high model capacity by dynamically selecting a subset of experts for each input label. However, despite these advantages, deploying MoE models in untrusted environments raises privacy concerns. On one hand, servers possessing the model weights expect their weights to be privacy-protected, as training a hybrid expert model requires significant financial and computational investment. On the other hand, the inference process of a hybrid expert model requires clients to upload their prompts, which may contain sensitive user data such as personal health records and biometric information. Therefore, servers require clients to be completely unaware of the model weights except for the inference results; while clients, in turn, require servers to be unaware of their inputs.

[0003] To address the privacy concerns in two-party neural network inference, some early studies explored privacy-preserving inference methods using secure multi-party computation. Some hybrid approaches, such as CrypTFlow2 and Cheetah, combine privacy-preserving inference with homomorphic encryption and Beaver triple-based additive secret sharing, achieving significant efficiency improvements.

[0004] In recent years, a significant amount of research has focused on two-party secure inference mechanisms tailored to the Transformer architecture. In the linear layer domain, Iron has improved upon the foundational framework established by Cheetah, extending the encoding paradigm from matrix-vector multiplication to include matrix-matrix multiplication. BumbleBee further compresses multiple output ciphertexts, thereby optimizing computational complexity and communication overhead. Recently, BOLT has implemented compact homomorphic ciphertext packing for linear layers using SIMD encoding, as well as polynomial approximation preprocessing for nonlinear functions. Nimbus proposed a client-side outer product protocol to pre-encrypt model weights during preprocessing, reducing overhead in the online phase. One of the current state-of-the-art protocols, SHAFT, relies on a trusted third party to implement a constant-round secure Softmax and GeLU protocol and integrates with the Hugging Face library, achieving privacy inference times in the sub-second range. Existing PPML solutions generally use basic Transformer models such as BERT and GPT-2, which are sufficient for early Transformer testing. However, they still have a certain gap in parameter scale compared with the plaintext models used in practice. For example, the number of parameters in existing PPML models is hundreds of times smaller than that of the open-source model Deepseek-V3.

[0005] To address the parameter expansion problem mentioned above, it is crucial to develop privacy-preserving protocols for parameter expansion, and one of the most mature technologies to consider is the Hybrid Expert Model (MoE).

[0006] In large language models employing the Transformer architecture, the hybrid expert layer consists of a set of... N exp A network of experts is composed of { FFN 1,…, FFN Nexp}, and supplemented by a gating network G Gating network G The input is assigned to the most suitable expert subnetwork for computation, and finally aggregated by combining the weights. This is based on the gating network. G Based on the strategy, MoE layers can be divided into two categories: dense MoE and sparse MoE.

[0007] Sparse hybrid expert models (MoEs) refer to models where a subset of expert subnetworks is selectively activated within each MoE layer. This is achieved through computation... K exp The weighted aggregation of expert outputs reduces computational costs significantly compared to the Dense MoE model.

[0008] However, directly implementing MoE using secure multi-party computation leads to privacy and performance issues. Previous research has utilized homomorphic encryption to compute linear layers, where the plaintext weight matrix is ​​held by the server. While sparse hybrid expert models require each token to activate only one layer... K exp There are multiple experts, but when plaintext expert weights are selected before computation, the server knows which weight matrices are accessed, thus leaking the client's private token information. A simple idea is to securely evaluate all experts using plaintext weight matrices and then inadvertently select experts. However, this method invalidates the sparsity of the mixed expert layer, requiring additional computation. N exp - K exp The presence of redundant experts leads to significant performance overhead, which is precisely what sparse hybrid expert models aim to avoid. Furthermore, the nonlinear functions included in the MoE hierarchy cause substantial performance overhead, necessitating the design of more efficient secure multi-party computation protocols to reduce communication rounds and total communication volume. Summary of the Invention

[0009] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a risk prevention and control method based on secure multi-party computation and hybrid expert models.

[0010] To achieve the above objectives, the present invention adopts the following technical solution: One aspect of the present invention provides a risk prevention and control method based on secure multi-party computation and hybrid expert models, comprising the following steps: Client C Holding a secret share of Addition [ x ] c ,server S Holding a secret share of Addition [ x ] s Key sk and the set of feedforward network weight matrices , i ∈[ N exp ], m , n and N exp These are the model dimensions, hidden dimensions, and total number of experts; Selection Phase: Client C and server S The selection and compression of the dense parameters are accomplished by calculating the secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vector; Computation phase: Client C and server S Calculation based on the dense state parameters obtained during the selection phaseK exp One expert and output.

[0011] As a preferred technical solution, the client C and server S The selection vector under secret sharing is calculated using the following steps: Client C and server S calculate π Topk Obtain activation values ​​for each dimension g ( x j )Central front K exp Find the maximum value and output the secret shared index. SortVal ];in x j For the first j The input to a feedforward network; Client C and server S calculate π onehot ([ SortVal ], K exp Output Boolean vector t b ∈{0,1} Nexp where the index position In other cases, the value is 0; Client C and server S calculate π B2A Used to input secrets t b Convert from Boolean form to arithmetic form and output the arithmetic form. t a The share value; Client C calculate and send to the server S ;in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation; server S The homomorphic ciphertext of the selection vector is calculated. .

[0012] As a preferred technical solution, the client C and server S The secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vectors are calculated, specifically including the following steps: server S calculate , and The homomorphic ciphertext of the accumulated weight group is obtained. , and Complete the selection of dense state parameters and compression; among which π MatMul To ensure element-wise safe multiplication, To select the homomorphic ciphertext of the vector.

[0013] As a preferred technical solution, the client C and server S Calculation based on the dense state parameters obtained during the selection phase Kexp Each expert will provide the following output: The overall process is as follows: parallel computing K exp Second-rate FFN = W r 2 ( σ ( W r 1 x )⨂ V r x ), K exp The homomorphic ciphertext of the chosen vector is different in each iteration; FFN The output of the feedforward network, W r 2 , W r 1 and V r For the cumulative result weighting reorganization, σ yes GeLU Activation function x =[ x ] c +[ x ] s This serves as the input for the entire method; The specific process is as follows: S1, Parallel Processing W r 1 x and V r xThe client homomorphically encrypts the shared value and sends it to the server. The server performs a ciphertext-to-ciphertext homomorphic multiplication locally, masks it with a random value, and sends it back to the client. The client decrypts the result to obtain the shared value. S2. Calculate the activation value of the secure activation function. σ ( W r 1 x ), and calculate the activation value and V r x The element multiplied by the element is obtained GLU The share value; S3, Calculation W r 2 GLU To obtain the final output π SparseMoE(x) =[ y ] c +[ y ] s in,[ y ] c For the client C , [ output share, y ] s For server S The output share.

[0014] As a preferred technical solution, the parallel processing W r 1 x and V r x Specifically: Client C For shared values ​​[ x ] c Encryption is performed to obtain the homomorphic ciphertext of the shared value. and send to the server S ;server S After receiving, perform local calculations. ,in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation; Server S calculates and Then combine with a random mask R 1 , R V calculate and ; serverS Will and Send to the client C and set ([ x GLU ] s ,[ x V ] s )=( R 1 , R V ); Client C Decrypt to obtain the result shared value ([ x GLU ] c ,[ x V ] c )=( W r 1 • x - R 1 , V r • x - R V ).

[0015] As a preferred technical solution, the computational security activation function σ ( W r 1 x ), and calculate the activation value and V r x The element multiplied by the element is obtained GLU The sharing value is as follows: Client C and server S calculate[ act ]:= π GeLU ([ x GLU ]), and obtained [ act ] c and[ act ] s ; Client C and server S Calculate point-by-point multiplication π MatMul ([ act ] i ,[ x V ] i ), to obtain the clientC and server S Holding separately GLU Share value[ GLU ] c and[ GLU ] s .

[0016] As a preferred technical solution, the calculation W r 2 GLU Specifically: Client C For holding GLU Share value[ GLU ] c Encrypt: and send to the server S ;server S Local computation ,in, This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation, [ GLU ] s For server S Holding GLU The share value; server S Calculate ciphertext-ciphertext multiplication , which uses and ; server S Combine random numbers R 2 calculate and send it to the client. C ;server S Will R 2 Set as output share[ y ] s ; Client C right Decrypt and W r 2 • GLU - R 2 Set as output share[ y ] c .

[0017] As a preferred technical solution, a single expert is selected for output, that is... K exp =1.

[0018] Another aspect of the present invention provides a risk prevention and control system based on secure multi-party computation and hybrid expert models, applied to the aforementioned risk prevention and control method based on secure multi-party computation and hybrid expert models, including a client. C ,server S The module includes a selection module, a calculation module, and a result output module. The client C and server S The selection module calculates the secure product and accumulation of the homomorphic ciphertext of the weight matrix set and the selection vector to complete the selection and compression of the dense parameters; The client C and server S The calculation module calculates based on the dense state parameters. K exp One expert; The result output module is used to output the results.

[0019] In another aspect, the present invention also provides a storage medium storing a program that, when executed by a processor, implements the above-described risk prevention and control method based on secure multi-party computation and hybrid expert models.

[0020] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) At the model level, this invention eliminates at least [missing information] by performing a selection phase. N exp - K exp Redundant computations by unselected experts are performed while maintaining sparsity under the MoE setting. The resulting overhead primarily comes from encrypted matrix multiplications during the computation phase. In fact, this is related to reducing... N exp - K exp These overheads are relatively small compared to the amount of computation saved by each expert.

[0021] (2) At the expert level, compared with BumbleBee, one of the current best solutions, this solution needs to... K Transformation protocol on one-hot vector π B2A Its cost is B2A It is also necessary Communication and local π MatMul In the computation phase, it differs from BumbleBee's computation. This scheme requires three ciphertext multiplications, that is... , and However, compared to the cost dominated by redundant computations from all experts, the additional cost of ciphertext multiplication is fixed and relatively small. Therefore, the effectiveness of this optimization becomes increasingly significant as the number of experts increases, reaching up to 29.8 times in tests. Attached Figure Description

[0022] cost The diagram shows the structure of the existing working MoE calculation process (left figure) and the risk prevention and control method based on secure multi-party computation and hybrid expert model in this embodiment of the invention (right figure). Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0024] Sparse hybrid expert models (MoEs) refer to models where a subset of expert subnetworks is selectively activated within each MoE layer. This is achieved through computation... K exp The weighted aggregation of expert outputs results in a significant reduction in computational overhead compared to dense hybrid expert models (Dense MoE). ; ; in g ( x )express Figure 1 Input for the operation g ( x ) exp It was before K exp Each element. Hyperparameters K exp It represents the number of experts selected.

[0025] This scheme uses a lattice-based additive homomorphic encryption scheme to construct the linear layer. Homomorphic encryption allows one party to compute encrypted data from another party without needing a decryption key. This homomorphic encryption scheme converts plaintext vectors... Encoded as plaintext polynomial ,Then Encrypted into ciphertext ,in q It is the ciphertext modulus. Therefore, the input in the linear layer is of dimension . k ×m of The server holding dimension is m × n plaintext weight Then the encoding function π L and π R The input will be encoded as a polynomial. and : , i ∈[ k ], j ∈[ m ], , i ∈[ m ], j ∈[ n ].

[0026] Example 1: like Softmax As shown, this embodiment provides a risk prevention and control method based on secure multi-party computation and a hybrid expert model. In the secure sparse hybrid expert protocol of this scheme, the client... C and server S Each holds a share of the input addition secret. x ] c and[ x ] s , x =[ x ] c +[ x ] s This is the input to the entire method, and x ={ x 0,…, x j ,…, x n-1}, j ∈[ n ], n This refers to the hidden layer dimension. Additionally, the server... S It also holds the plaintext feedforward network (FFN) weight matrix set for all experts. , i ∈[ N exp ], m , n and N exp These are the model dimensions, hidden dimensions, and total number of experts; simultaneously, the server... S Also holds the key Figure 1(Used for subsequent homomorphic encryption used on the server side, not in homomorphic operations).

[0027] I. Selection Phase: Client C and server S The selection and compression of the dense parameters are accomplished by calculating the secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vector.

[0028] During the selection phase, this scheme leverages the advantage of homomorphic encryption's local computation without communication, requiring the sharing of a length of... N exp The selection vector enables the unintentional selection of encryption weights, specifically including the following steps: S1, Client C and server S Calculate the selection vector under secret sharing: Client C and server S calculate π Topk ( π (Refers to functions, with subscripts indicating their functions and uses, the same below) to obtain activation values ​​for each dimension. g ( x i )Central front K exp Find the maximum value and output the secret shared index used to implement the unintentional selection of cryptographic weights. sk ];in x j This serves as the input to the feedforward network; Specifically, in the transformer model, each encoder has a MoE FFN, and the MoE FFN includes functions to determine its input. x j The matching expert, and the corresponding expert's calculation. x j It is the first j The input of MoE FFN is the output of the previous LayerNorm.

[0029] Client C and server S calculate π onehot ([ SortVal ], K exp Output Boolean vector t b ∈{0,1} Nexp where the index position In other cases, the value is 0; among them, SortValGiven a one-hot vector, input the secret sharing index and... K exp The length of the output secret sharing is K exp A Boolean vector, wherein only the index position of the Boolean vector is 1, and the rest are 0.

[0030] Client C and server S calculate π B2A Used to input secrets t b Convert from Boolean form to arithmetic form and output the arithmetic form. t a The shared value; where the input of the B2A algorithm is the Boolean vector shared value from the previous step (boolean, so it is B), and the output is the arithmetic form of the vector shared value (arithmetic, so it is A); the shared value means that each party holds a vector, and only by adding the two shared values ​​bit by bit can the true vector be recovered; Client C calculate and send to the server S ;in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. onehot (•) indicates a homomorphic encryption operation; server S The homomorphic ciphertext of the selection vector is calculated. .

[0031] S2, client C, and server S calculate the secure product and summation of the weight matrix set and the homomorphic ciphertext of the selection vector: server S calculate , and The homomorphic ciphertext of the accumulated weight group is obtained. , and Complete the selection of dense state parameters and compression; among which π MatMul To ensure element-wise safe multiplication, To select the homomorphic ciphertext of the vector.

[0032] II. Calculation Phase: Client C and server S Calculation based on the dense state parameters obtained during the selection phase K exp One expert.

[0033] During the computation phase, the computation method of this scheme is similar to previous studies. The key difference is that the output of the selection phase is homomorphic ciphertext to prevent the leakage of selection information to the server, while past works typically used plaintext. Although this introduces the additional computational overhead of homomorphic ciphertext-ciphertext matrix multiplication, this scheme only calls this multiplication operator twice, compared to the computation of the remaining... N exp - K exp Compared to an expert FFN, it is more efficient.

[0034] S3, Client C and server S Calculation based on the dense state parameters obtained during the selection phase K exp The process involves multiple experts providing output, and the overall workflow is as follows: parallel computing K exp Second-rate Enc = W r 2 ( FFN ( W r 1 x )⨂ V r x ), K exp The homomorphic ciphertext of the selection vector is different in each step (corresponding to experts with different weights). σ The output of the feedforward network, W r 2 , W r 1 and V r For the cumulative result weighting reorganization, FFN yes σ Activation function x =[ x ] c +[ x ] s This serves as the input for the entire method; in the parallel computation, each time... K exp Incrementing sequentially, selecting the first [item] in parallel K exp A relatively large value; subsequent calculations are similar.

[0035] The specific process is as follows: S3.1 Parallel Processing W r 1 xand V r x The client homomorphically encrypts the shared value and sends it to the server. The server performs a homomorphic multiplication of the ciphertext and sends it back to the client after masking it with a random value. The client decrypts the result to obtain the shared value.

[0036] Furthermore, step S3.1 specifically includes: Client C For shared values ​​[ x ] c Encryption is performed to obtain the homomorphic ciphertext of the shared value. and send to the server S ;server S After receiving, perform local calculations. ,in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. GeLU (•) indicates a homomorphic encryption operation; Server S calculates and Then combine with a random mask R 1 , R V calculate and ; server S Will and Send to the client C and set ([ x GLU ] s ,[ x V ] s )=( R 1 , R V ); Client C Decrypt to obtain the result shared value ([ x GLU ] c ,[ x V ] c )=( W r 1 • x - R 1 , V r • x - R V ).

[0037] S3.2 Calculate the activation value of the secure activation function. Enc ( W r 1 x ), and calculate the activation value and V r x The element multiplied by the element is obtained σ The share value.

[0038] Furthermore, step S3.2 specifically includes: Client C and server S calculate[ GLU ]:= π GeLU ([ x GLU ]), and obtained [ act ] c and[ act ] s ; Client C and server S Calculate point-by-point multiplication π MatMul ([ act ] i ,[ x V ] i ), to obtain the client C and server S Holding separately act Share value[ GLU ] c and[ GLU ] s .

[0039] S3.3, Calculation W r 2 GLU To obtain the final output π SparseMoE(x) =[ y ] c +[ y ] s in,[ y ] c For the client C , [ output share, y ] s For server S The output share.

[0040] Furthermore, step S3.3 specifically includes: Client CFor holding GLU Share value[ GLU ] c Encrypt: and send to the server S ;server S Local computation ,in, This represents the ciphertext in a homomorphic encryption form; := indicates assignment. GLU (•) indicates a homomorphic encryption operation, [ Enc ] s For server S Holding GLU The share value; server S Calculate ciphertext-ciphertext multiplication , which uses and ; server S Combine random numbers R 2 calculate and send it to the client. C ;server S Will R 2 Set as output share[ y ] s ; Client C right Decrypt and W r 2 • GLU - R 2 Set as output share[ y ] c .

[0041] III. Output.

[0042] Example 2: This embodiment selects only a single expert for output, that is, let K exp =1, minimizing redundant calculations by unselected experts. The remaining steps are the same as in Example 1.

[0043] Example 3: In this embodiment, a risk prevention and control system based on secure multi-party computation and hybrid expert models is provided. The system includes a client. C ,server S The module includes a selection module, a calculation module, and a result output module. The client C and serverS The selection module calculates the secure product and accumulation of the homomorphic ciphertext of the weight matrix set and the selection vector to complete the selection and compression of the dense parameters; The client C and server S The calculation module calculates based on the dense state parameters. K exp One expert; The result output module is used to output the results.

[0044] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above. The system can be applied to a risk prevention and control method based on secure multi-party computation and hybrid expert model in the above embodiments.

[0045] Example 4: In this embodiment, a storage medium is also provided, storing a program. When the program is executed by a processor, it implements a risk prevention and control method based on secure multi-party computation and a hybrid expert model as described in the above embodiment, specifically: Protocol input: Client C and server S Each holds a share of the input addition secret. x ] c and[ x ] s , x =[ x ] c +[ x ] s This is the input to the entire method, and x ={ x 0,…, x j ,…, x n-1}, j ∈[ n ], n This refers to the hidden layer dimension. Additionally, the server... S It also holds the plaintext feedforward network (FFN) weight matrix set for all experts. , i ∈[ N exp ], m , n and N exp These are the model dimensions, hidden dimensions, and total number of experts; simultaneously, the server... SAlso holds the key GLU sk .

[0046] Selection Phase: Client C and server S The selection and compression of the dense parameters are accomplished by calculating the secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vector.

[0047] Computation phase: Client C and server S Calculation based on the dense state parameters obtained during the selection phase K exp One expert and output.

[0048] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0049] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A risk prevention and control method based on secure multi-party computation and hybrid expert models, characterized in that, Includes the following steps: Client C Holding a secret share of Addition [ x ] c ,server S Holding a secret share of Addition [ x ] s Key sk and the set of feedforward network weight matrices , i ∈[ N exp ], m , n and N exp These are the model dimensions, hidden dimensions, and total number of experts; Selection Phase: Client C and server S The selection and compression of the dense parameters are accomplished by calculating the secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vector; Computation phase: Client C and server S Calculation based on the dense state parameters obtained during the selection phase K exp One expert and output.

2. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 1, characterized in that, Client C and server S The selection vector under secret sharing is calculated using the following steps: Client C and server S calculate π Topk Obtain activation values ​​for each dimension g ( x j )Central front K exp Find the maximum value and output the secret shared index. SortVal ];in x j For the first j The input to a feedforward network; Client C and server S calculate π onehot ([ SortVal ], K exp Output Boolean vector t b ∈{0,1} Nexp where the index position In other cases, the value is 0; Client C and server S calculate π B2A Used to input secrets t b Convert from Boolean form to arithmetic form and output the arithmetic form. t a The share value; Client C calculate and send to the server S ;in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation; server S The homomorphic ciphertext of the selection vector is calculated. .

3. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 1, characterized in that, Client C and server S The secure product and accumulation of the weight matrix set and the homomorphic ciphertext of the selection vectors are calculated, specifically including the following steps: server S calculate , and The homomorphic ciphertext of the accumulated weight group is obtained. , and Complete the selection of dense state parameters and compression; among which π MatMul To ensure element-wise safe multiplication, To select the homomorphic ciphertext of the vector.

4. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 1, characterized in that, The client C and server S Calculation based on the dense state parameters obtained during the selection phase Kexp Each expert will provide the following output: The overall process is as follows: parallel computing K exp Second-rate FFN = W r 2 ( σ ( W r 1 x )⨂ V r x ), K exp The homomorphic ciphertext of the chosen vector is different in each iteration; FFN The output of the feedforward network, W r 2 , W r 1 and V r For the cumulative result weighting reorganization, σ yes GeLU Activation function x =[ x ] c +[ x ] s This serves as the input for the entire method; The specific process is as follows: S1, Parallel Processing W r 1 x and V r x The client homomorphically encrypts the shared value and sends it to the server. The server performs a ciphertext-to-ciphertext homomorphic multiplication locally, masks it with a random value, and sends it back to the client. The client decrypts the result to obtain the shared value. S2. Calculate the activation value of the secure activation function. σ ( W r 1 x ), and calculate the activation value and V r x The element multiplied by the element is obtained GLU The share value; S3, Calculation W r 2 GLU To obtain the final output π SparseMoE (x) =[ y ] c +[ y ] s in,[ y ] c For the client C , [ output share, y ] s For server S The output share.

5. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 4, characterized in that, The parallel processing W r 1 x and V r x Specifically: Client C For shared values ​​[ x ] c Encryption is performed to obtain the homomorphic ciphertext of the shared value. and send to the server S ; server S After receiving, perform local calculations. ,in This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation; Server S calculates and Then combine with a random mask R 1 , R V calculate and ; server S Will and Send to the client C and set ([ x GLU ] s ,[ x V ] s )=( R 1 , R V ); Client C Decrypt to obtain the result shared value ([ x GLU ] c ,[ x V ] c )=( W r 1 • x - R 1 , V r • x - R V ).

6. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 4, characterized in that, The computational security activation function σ ( W r 1 x ), and calculate the activation value and V r x The element multiplied by the element is obtained GLU The sharing value is as follows: Client C and server S calculate[ act ]:= π GeLU ([ x GLU ]), and obtained [ act ] c and[ act ] s ; Client C and server S Calculate point-by-point multiplication π MatMul ([ act ] i ,[ x V ] i ), to obtain the client C and server S Holding separately GLU Share value[ GLU ] c and[ GLU ] s .

7. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to claim 4, characterized in that, The calculation W r 2 GLU Specifically: Client C For holding GLU Share value[ GLU ] c Encrypt: and send to the server S ; server S Local computation ,in, This represents the ciphertext in a homomorphic encryption form; := indicates assignment. Enc (•) indicates a homomorphic encryption operation, [ GLU ] s For server S Holding GLU The share value; server S Calculate ciphertext-ciphertext multiplication , which uses and ; server S Combine random numbers R 2 calculate and send it to the client. C ;server S Will R 2 Set as output share[ y ] s ; Client C right Decrypt and W r 2 • GLU - R 2 Set as output share[ y ] c .

8. The risk prevention and control method based on secure multi-party computation and hybrid expert model according to any one of claims 1-7, characterized in that, Choose a single expert for output, that is... K exp =1.

9. A risk prevention and control system based on secure multi-party computation and hybrid expert models, characterized in that, The risk control method based on secure multi-party computation and hybrid expert model, applied to any one of claims 1-7, includes a client. C ,server S The module includes a selection module, a calculation module, and a result output module. The client C and server S The selection module calculates the secure product and accumulation of the homomorphic ciphertext of the weight matrix set and the selection vector to complete the selection and compression of the dense parameters; The client C and server S The calculation module calculates based on the dense state parameters. K exp One expert; The result output module is used to output the results.

10. A storage medium storing a program, characterized in that: When the program is executed by the processor, it implements the risk prevention and control method based on secure multi-party computation and hybrid expert model as described in any one of claims 1-7.

Images