An LLM collaborative inference system based on privacy protection adversarial fine-tuning

Abstract

The application provides a LLM collaborative inference system based on privacy protection adversarial fine-tuning, comprising a model splitting deployment module, which is used for splitting a pre-trained large language model into a shallow front sub-model and a shallow rear sub-model deployed on a client, and a deep middle sub-model hosted on a cloud server; a double-layer constraint adversarial fine-tuning module embedded in the shallow front sub-model and the shallow rear sub-model of the client, comprising a customized adapter module, which is used for modifying the structure and parameters of the client model under specified conditions to resist potential reconstruction attacks; wherein the adversarial fine-tuning module introduces an auxiliary privacy constraint mechanism and an accuracy constraint condition; and a collaborative inference model, which is used for collaborative inference by the client and the cloud server after completing the adversarial fine-tuning processing. The framework of the application effectively balances privacy protection, inference utility and computing efficiency without any modification of the server-side model structure and parameters.

Description

Technical Field

[0001] This invention relates to the field of cyberspace security technology, specifically to an LLM collaborative inference system based on privacy protection and adversarial fine-tuning, which aims to solve the problem of user privacy leakage faced by large-scale language models deployed in the cloud during the inference process. Background Technology

[0002] In recent years, large-scale language models (LLMs) based on the Transformer architecture have made significant progress in the field of natural language processing. Models such as Generative Pre-trained Transformer (GPT) have been widely used in education, creative writing, medical care, and scientific research, driving significant performance improvements. Currently, large-scale language models are mainly deployed as cloud services, with service providers hosting the models on cloud servers and providing inference capabilities to users through API interfaces.

[0003] However, this cloud-based deployment method inherently brings privacy risks and potential data breaches. Users must upload their inputs to the server in plaintext to obtain inference results, and this input may contain sensitive information (such as medical records in a healthcare setting) or the user's own original content (such as edited text submitted in a text editing application). Although numerous privacy-preserving methods for large-scale language model inference have been proposed in recent years, most methods still have many limitations: Cryptographic-based solutions require large-scale modifications to the entire server model, which not only significantly limits their practicality but also introduces serious performance bottlenecks.

[0004] Differential privacy (DP) based schemes typically employ random replacement strategies or directly inject Gaussian noise into the embedding matrix to achieve privacy protection through fine-tuning and inference on the noisy input. However, these methods often lead to a decline in inference performance, and a trade-off must be made between privacy and utility.

[0005] User-specific key encryption scheme: The entire server model needs to be fine-tuned for the user. In multi-user scenarios, the server must maintain and deploy an independent model for each user, which greatly increases deployment and storage costs.

[0006] In summary, existing methods suffer from severe computational inefficiencies and inevitably fall into a trade-off between privacy and utility. Their core limitation lies in the extensive modification of server-side model architecture and parameters, which not only brings great complexity to system deployment and maintenance but also generates huge computational overhead. At the same time, the need to adapt the model for each user further restricts the scalability and practical feasibility of these solutions in real-world scenarios. Summary of the Invention

[0007] To address the various shortcomings of existing technologies, the present invention aims to provide an LLM collaborative inference system based on privacy protection and adversarial fine-tuning. This system framework effectively balances privacy protection, inference utility, and computational efficiency without requiring any modification to the server-side model structure and parameters.

[0008] The present invention achieves the above objectives through the following technical solutions: A privacy-preserving adversarial fine-tuning-based LLM collaborative reasoning system includes: The model splitting and deployment module is used to split a pre-trained large language model into shallow pre-sub-models and shallow post-sub-models deployed on the client, and deep intermediate sub-models hosted on the cloud server; wherein, the client and the cloud server achieve collaborative reasoning by exchanging processed intermediate representations. The two-layer constraint adversarial fine-tuning module is embedded in the shallow pre-sub-model and shallow post-sub-model of the client. It includes a customized adapter module for modifying the structure and parameters of the client model under specified conditions to resist potential reconstruction attacks. The adversarial fine-tuning module introduces an auxiliary privacy constraint mechanism to maximize the distribution difference between the encryptor model output and the original pre-sub-model output, while combining it with precision constraints to ensure that the model inference efficiency is maintained while protecting privacy. The collaborative reasoning model, after completing the adversarial fine-tuning process, enables the client and cloud server to perform collaborative reasoning. The collaborative reasoning model includes a client encryptor model, a client decryptor model, and a deep intermediate sub-model, all processed by a two-layer constraint adversarial fine-tuning module.

[0009] According to the LLM collaborative inference system based on privacy-preserving adversarial fine-tuning provided by the present invention, in the model splitting and deployment module, the large language model held by the cloud server CS... Including word embedding encoding module, A Transformer decoder layer and a word embedding decoding module are used to provide inference services to the client CL; The cloud server CS divides the model into three segments according to the hierarchical structure: the shallow sub-model. Intermediate sub-model With shallow sub-model ,in Pre-split points are specified for the client CL and communicated to the cloud server CS; Among them, the shallow pre-sub-model Including word embedding encoding module and front One Transformer decoder layer; intermediate submodel Including the To the Layered Transformer decoder; shallow sub-model Including the To the The Transformer decoder for the layer and the word embedding decoding module.

[0010] According to the present invention, an LLM collaborative inference system based on privacy-preserving adversarial fine-tuning includes the following deployment method and inference process for model split deployment modules: Cloud server CS retains intermediate sub-model and shallow pre-sub-model With shallow sub-model Send it to the client CL for local deployment, including the shallow pre-sub-model. With shallow sub-model The model size is smaller than the intermediate sub-model. ; When performing collaborative reasoning, given the input embedding of the client CL... The collaborative reasoning process can be represented by the following formula:

[0011] Among them, the client-side CL utilizes a shallow pre-sub model. For input x The process is performed to generate a first intermediate representation, which is then transmitted to the cloud server CS. The cloud server CS utilizes the intermediate sub-model. The first intermediate representation is processed to generate a second intermediate representation, which is then transmitted back to the client CL. The client CL utilizes the shallow post-submodel. The second intermediate representation is processed to obtain the final output. y .

[0012] According to the present invention, an LLM collaborative inference system based on privacy-preserving adversarial fine-tuning is provided, wherein the client CL introduces a customized adapter module into the shallow pre-sub-model and shallow post-sub-model of the client. To implement modifications to the client-side model, where Indicates the first The parameters of each adapter module, each adapter module adopts a bottleneck neural network structure, consisting of dimensions... Input and output layers Each dimension is (and It consists of a hidden layer and residual connections connecting the input layer and the output of the hidden layer; The shallow pre-submodel will be deployed on the client. After embedding the adapter module, it is defined as an encryptor model. The shallow post-submodel will be deployed on the client. After embedding the adapter module, it is defined as a decryptor model. ,in, This represents the set of parameters for all adapter modules; After the adapter embedding is completed, the model inference formula for the collaborative inference process is updated as follows:

[0013] in, For the encryption model to input to the client x The processing results This is an intermediate sub-model for cloud servers. ( ) represents the decoding result output by the decryptor model to the intermediate sub-model.

[0014] According to the LLM collaborative reasoning system based on privacy-preserving adversarial fine-tuning provided by the present invention, in the two-layer constraint adversarial fine-tuning module, the client sub-model and They were respectively converted into encryption model and decryptor model ; The two-layer constraint adversarial fine-tuning module employs a two-layer constraint adversarial fine-tuning strategy, including: Client-side CL and cloud server CS in auxiliary dataset Collaborative cryptographic model and decryptor model Adjust the parameters; During the fine-tuning process, the intermediate sub-model deployed on the cloud server CS The parameters remain frozen and do not participate in parameter updates; During fine-tuning, both accuracy and privacy constraints are simultaneously met: Precision constraints are used to quantify the difference between the fine-tuned collaborative inference output and the original pre-trained model output. Privacy constraints are used to quantify the distributional differences between the output of the encryptor model and the original output of the former sub-model.

[0015] According to the LLM collaborative reasoning system based on privacy-preserving adversarial fine-tuning provided by the present invention, the collaborative reasoning module performs the following forward propagation steps: During the forward propagation process, the client CL first uses the Word2Vector module to perform word segmentation and vectorization on the batch text sequence to generate batch input. ,in, The length of the text represented by a fixed-length sequence of token IDs. Indicates the batch size of the input text; Then, the client CL will input via encryptor model Compared with shallow pre-sub-model Processing to generate encrypted intermediate representations With the original The client CL transmits through an encrypted channel and Securely transmitted to the cloud server CS; After receiving the data, the cloud server CS uses an intermediate sub-model. Process these two sets of intermediate representations separately to obtain the inference output: Intermediate output of the encrypted path: ; Intermediate output of the original path: ; And and Safely return to the client CL; Finally, the client CL uses the decryptor model respectively. With shallow sub-model right and The process is then performed to generate the final collaborative reasoning output: Encrypted path output logits: ; Original path output logits: ; in, Represents the pre-trained model The size of the vocabulary.

[0016] According to the present invention, an LLM collaborative inference system based on privacy-preserving adversarial fine-tuning includes the following accuracy constraints for the two-layer constraint adversarial fine-tuning module: Kullback-Leibler divergence is used to quantify the collaborative inference output. Compared with the original pre-trained model output The difference in probability distributions between them defines the precision-preserving loss function. Its mathematical expression is:

[0017] in, This represents the softmax function, used to convert the model output into a probability distribution; Temperature is a commonly used temperature parameter in knowledge distillation, used to smooth the probability distribution, multiplied by This can offset the effect of temperature on gradient scaling; Indicates the batch size of the input text; Represents the pre-trained model The size of the vocabulary; NS The length of the token sequence is fixed. and The original model and the collaborative reasoning model were respectively in the 1st... b The first batch, the first s The token position, the first v The output logits at each vocabulary index.

[0018] According to the present invention, an LLM collaborative reasoning system based on privacy-preserving adversarial fine-tuning includes a privacy constraint mechanism for a two-layer constraint adversarial fine-tuning module, comprising: A privacy loss function based on cosine similarity is adopted. Quantization Encryptor Model intermediate representation of output Compared with shallow pre-sub-model intermediate representation of output The mathematical expression for the distribution difference between them is:

[0019] in, and The cipher model and the original pre-sub-model are respectively in the 1st... b The first batch, the first s The intermediate representation vector at each token position Represents the dot product of vectors, ∥ ||2 represents the L2 norm of the vector; Preserving accuracy loss function With privacy loss function Merge into one with adjustable parameters joint loss function Its expression is:

[0020] in, Adjustable parameters to balance accuracy constraints and privacy constraints, It is a small constant used to prevent gradient explosion when the privacy loss is minimal in the early stages of optimization.

[0021] According to the LLM collaborative inference system based on privacy-preserving adversarial fine-tuning provided by the present invention, the backpropagation and parameter update process of the two-layer constraint adversarial fine-tuning module includes: A gradient-based backpropagation method is employed, along with a joint loss function. As an optimization objective, the client's encryptor model is jointly optimized while satisfying accuracy and privacy constraints. and decryptor model ; In each iteration, the client determines the loss function based on the joint loss function. Calculate the gradients of the parameters of the two models, due to the output of co-inference. Only participate in the accuracy-preserving loss function First calculate The corresponding gradient, and through the decryptor model Backpropagation is performed to obtain the parameter gradients of the decryptor. , and intermediate characterization gradient ; The client computes its intermediate representation of the previous stage directly on its local machine. joint gradient :

[0022] in, For accuracy loss gradient, For privacy loss gradient, Adjustable parameters to balance accuracy and privacy, >0 is a small constant to prevent gradient explosion; Joint gradient Through the cryptographer model By performing backpropagation, the parameter gradients of the encryptor can be calculated. and ; Finally, the learning rate is used. The model parameters are updated, as expressed by the following formula: Decryptor model parameter update:

[0023] Encryptor model parameter update:

[0024] According to the LLM collaborative inference system based on privacy-preserving adversarial fine-tuning provided by the present invention, after completing the adversarial fine-tuning process, the client CL removes the original shallow pre-sub-model. With shallow sub-model Only the finely tuned encryptor model is retained. and decryptor model It performs collaborative reasoning together with the cloud server CS; First, the client-side CL uses a word embedding encoding module to convert the input text sequence into an input embedding. and will Input to the encryptor model Generate intermediate representations ; Subsequently, the client CL transmitted through a secure channel Send to cloud server CS; Cloud server CS received Then, the intermediate sub-model is frozen using parameters. right Processing is performed to obtain intermediate representations. Then it is sent back to the client CL; Finally, the client CL will Input to the decryptor model Together with the word embedding decoding module, a prediction token is generated and appended to the end of the current input sequence as the new input for the next round of autoregressive prediction; Repeat the steps of input embedding and encryption, intermediate representation transmission, intermediate sub-model processing, token generation and autoregressive iteration until the preset stopping condition is met, and finally output the generated complete text sequence.

[0025] Therefore, compared with the prior art, the LLM collaborative reasoning system based on privacy-preserving adversarial fine-tuning proposed in this invention has the following beneficial effects: 1. This invention, through model segmentation and collaborative inference architecture, divides the LLM into a shallow client-side sub-model and a deep cloud-based sub-model, transmitting only intermediate representations rather than the original input data, thus fundamentally avoiding the direct exposure of users' sensitive information. Simultaneously, it introduces a two-layer constraint adversarial fine-tuning (AdvFT) mechanism, dynamically adjusting client-side model parameters to cause irreversible changes in the distribution of intermediate representations. Even if an attacker obtains the intermediate data, they cannot reconstruct the original input, significantly enhancing privacy protection.

[0026] 2. Existing privacy protection solutions (such as fully homomorphic encryption and secure multi-party computation) require complex encryption of the model or data, resulting in inference latency increasing by tens of times. This invention deploys computationally intensive tasks (such as the Transformer layer) in the cloud through model partitioning. The client only needs to perform lightweight shallow computations. Combined with the AdvFT mechanism for fine-tuning and optimizing model parameters, the inference speed is greatly improved compared to traditional cloud deployment solutions, while reducing the loss of inference accuracy.

[0027] 3. This invention supports diverse LLM tasks such as autoregressive text generation, text classification, and question-answering systems through a unified intermediate representation exchange interface, eliminating the need for dedicated privacy protection modules for different tasks. For example, in a medical diagnosis scenario, the system can simultaneously handle medical record text classification (disease type determination) and generative responses (diagnosis suggestion generation), and the privacy protection strength for both tasks can be independently adjusted via the AdvFT parameters. Furthermore, the model splitting points can be dynamically configured to adapt to different hardware environments (such as mobile devices and edge devices), significantly improving the system's deployment flexibility.

[0028] 4. Traditional privacy protection solutions require structural modifications or retraining of cloud-based models, which are costly and difficult to implement. This invention achieves privacy protection through adversarial fine-tuning on the pure client side. The cloud-based model is directly compatible without any modifications and can be seamlessly integrated into existing mainstream LLM services such as GPT and BERT. For example, in commercial API services based on GPT-3, only the deployment of a shallow sub-model and the AdvFT module on the user's end is needed to achieve inference privacy protection, significantly lowering the technical implementation threshold.

[0029] 5. This invention utilizes the privacy constraint weight parameter λ in the AdvFT mechanism to allow users to dynamically balance privacy protection and inference utility according to scenario requirements. For example, in highly sensitive scenarios such as financial transactions, the value of λ can be increased to enhance privacy protection; in less sensitive scenarios such as content recommendation, the value of λ can be decreased to optimize user experience. This invention is widely applicable to privacy-sensitive scenarios in multiple fields such as healthcare, finance, and education.

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the architecture of an embodiment of the LLM collaborative reasoning system based on privacy protection and adversarial fine-tuning according to the present invention.

[0032] Figure 2 This is a flowchart illustrating an embodiment of an LLM collaborative reasoning system based on privacy protection and adversarial fine-tuning according to the present invention.

[0033] Figure 3 This is a schematic diagram of the adapter module structure and its insertion position in an embodiment of an LLM collaborative reasoning system based on privacy protection and adversarial fine-tuning according to the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0035] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0036] See Figure 1 This embodiment provides an LLM collaborative reasoning system based on privacy-preserving adversarial fine-tuning, including: The model splitting and deployment module is used to split a pre-trained large language model into shallow pre-sub-models and shallow post-sub-models deployed on the client, and deep intermediate sub-models hosted on the cloud server. The client and the cloud server achieve collaborative reasoning by exchanging processed intermediate representations, which effectively avoids the client directly transmitting plaintext input data containing sensitive information to the cloud server. The two-layer constraint adversarial fine-tuning module is embedded in the shallow pre-sub-model and shallow post-sub-model of the client. It includes a customized adapter module for modifying the structure and parameters of the client model under specified conditions to resist potential reconstruction attacks. The adversarial fine-tuning module introduces an auxiliary privacy constraint mechanism to maximize the distribution difference between the encryptor model output and the original pre-sub-model output, while combining it with precision constraints to ensure that the model inference efficiency is maintained while protecting privacy. The collaborative reasoning model, after completing the adversarial fine-tuning process, enables the client and cloud server to perform collaborative reasoning. The collaborative reasoning model includes a client encryptor model, a client decryptor model, and a deep intermediate sub-model, all processed by a two-layer constraint adversarial fine-tuning module.

[0037] As can be seen, this embodiment proposes a novel privacy-preserving inference framework (PI-SAFE), such as... Figure 1As shown, efficient, accurate, and privacy-preserving LLM inference can be achieved without any modification to the server-side model. Specifically, PI-SAFE splits a pre-trained large language model between the client and server, achieving efficient collaborative inference by transmitting encoded intermediate representations instead of plaintext input. To further enhance privacy protection and resist reconstruction attacks, an Adversarial Fine-Tuning (AdvFT) mechanism is introduced. This mechanism allows the client to modify the structure and parameters of its own model without the server's knowledge, thereby changing the output distribution while maintaining inference utility.

[0038] In cloud-based LLM inference deployment scenarios, the cloud server CS holds a pre-trained large-scale language model. Including word embedding encoding module, A Transformer decoder layer and a word embedding decoding module are used to provide inference services to the client CL; let the user input be... Then the output of LLM is .

[0039] The cloud server CS divides the model into three segments according to the hierarchical structure: the shallow sub-model. Intermediate sub-model With shallow sub-model ,like Figure 2 As shown, where Pre-split points are specified for the client CL and the cloud server CS is informed.

[0040] Specifically, shallow pre-sub-model Including word embedding encoding module and front One Transformer decoder layer; intermediate submodel Including the To the Layered Transformer decoder; shallow sub-model Including the To the The Transformer decoder for the layer and the word embedding decoding module.

[0041] As can be seen, this embodiment of PI-SAFE employs a segmentation strategy, dividing a large language model into three parts: a shallow front sub-model and a shallow rear sub-model deployed on the client, and a deep middle sub-model hosted in the cloud. This architecture achieves collaborative reasoning through the exchange of intermediate representations, effectively avoiding the need for the client to transmit plaintext input to the server, thereby significantly improving computational efficiency while maintaining reasoning utility.

[0042] In this embodiment, the deployment method and inference process of the model split deployment module include: Subsequently, the cloud server CS retains the intermediate sub-model. and smaller shallow pre-sub-model With shallow sub-model Send it to the client CL for local deployment, including the shallow pre-sub-model. With shallow sub-model The model size is smaller than the intermediate sub-model. At this point, the inference process of LLM needs to be completed collaboratively by CS and CL, and the two interact through intermediate representations.

[0043] Specifically, in collaborative reasoning, given the input embedding of the client CL... The reasoning for the entire model can be expressed by the following formula:

[0044] Among them, the client-side CL utilizes a shallow pre-sub model. For input x The process is performed to generate a first intermediate representation, which is then transmitted to the cloud server CS. The cloud server CS utilizes the intermediate sub-model. The first intermediate representation is processed to generate a second intermediate representation, which is then transmitted back to the client CL. The client CL utilizes the shallow post-submodel. The second intermediate representation is processed to obtain the final output. y ; Once the LLM is split and deployed on both the cloud server (CS) and the client-side client (CL), the CS gains access to the structure and parameters of the client-side CL sub-model. This allows the cloud server (CS) to launch a refactoring attack in a white-box environment and... Accurately reconstruct the original user input To defend against refactoring attacks, CL must render CS completely unaware of its structure and parameters, thus breaking the preconditions for refactoring attacks.

[0045] In this embodiment, drawing on an adapter-based fine-tuning approach, the client CL introduces a customized adapter module into the shallow pre-sub-model and shallow post-sub-model of the client. To achieve this modification, where Indicates the first The parameters of each adapter module, the architecture of the adapter module, and its insertion position are as follows: Figure 3 As shown. Each adapter module adopts a bottleneck neural network structure, consisting of a dimension of... Input and output layers Each dimension is (and It consists of a hidden layer and a residual connection connecting the input layer and the output of the hidden layer, and the number of parameters is controlled by a bottleneck structure.

[0046] The adapter module is typically inserted after the attention layer and feedforward layer of the local sub-model, or directly cascaded to the input of the Transformer decoder block, where the parameters... depending on The embedding dimension, and Then it is randomly specified by the user.

[0047] The shallow pre-submodel will be deployed on the client. After embedding the adapter module, it is defined as an encryptor model. The shallow post-submodel will be deployed on the client. After embedding the adapter module, it is defined as a decryptor model. ,like Figure 2 As shown on the right. Among them, This represents the set of parameters for all adapter modules.

[0048] Therefore, after the adapter embedding is completed, the model inference formula for the collaborative inference process is updated as follows:

[0049] in, For the encryption model to input to the client x The processing results This is an intermediate sub-model for cloud servers. ( ) represents the decoding result output by the decryptor model to the intermediate sub-model.

[0050] In the two-layer constraint adversarial fine-tuning module, following the steps described above, the client sub-model... and They were respectively converted into encryption model and decryptor model .

[0051] However, the integration of the adapter module may introduce biases in the collaborative inference output, leading to... To mitigate this bias, the two-layer constraint adversarial fine-tuning module employs a two-layer constraint adversarial fine-tuning strategy (AdvFT), which includes: Client-side CL and cloud server CS in auxiliary dataset Collaborative cryptographic model and decryptor model Adjust the parameters; During the fine-tuning process, the intermediate sub-model deployed on the cloud server CS The parameters remain frozen and do not participate in parameter updates; The fine-tuning process simultaneously satisfies both accuracy and privacy constraints: Precision constraints are used to quantify the difference between the fine-tuned collaborative inference output and the original pre-trained model output, ensuring inference utility. Privacy constraints are used to quantify the distribution difference between the output of the encryptor model and the original output of the sub-model, thereby maximizing this difference to resist refactoring attacks on cloud servers. In this process, the intermediate sub-model deployed on the cloud server The parameters remain frozen. The data flow during the AdvFT process is as follows: Figure 2 As shown.

[0052] As can be seen, through the aforementioned two-layer constraint adversarial fine-tuning, the client can modify the structure and parameters of its own model without the server's knowledge, mitigating the collaborative inference output bias introduced by the adapter module integration and achieving a balance between privacy protection and inference accuracy. Specifically, the client embeds customized adapter modules in the front and back sub-models respectively, transforming them into encryptor and decryptor models. During the fine-tuning process, auxiliary privacy constraints are introduced to maximize the distribution difference between the encryptor model output and the original front sub-model output, thereby changing the model structure and parameters and rendering the adversary's existing architectural priors invalid. (Even if the adversary fully understands the original front sub-model, it cannot accurately reconstruct the input from the observed intermediate representations, because these representations have been generated by the modified encryptor model, and the input-to-output mapping has been fundamentally changed.) At the same time, to maintain inference utility, the fine-tuning process must also meet accuracy constraints, ensuring that the server-side intermediate sub-models are not modified and that the inference results maintain high fidelity. In addition, this mechanism supports adjustable privacy protection strength, allowing users to customize model segmentation positions and fine-tune hyperparameters according to individual needs.

[0053] Therefore, PI-SAFE achieves adjustable privacy protection, efficient inference, and minimal utility loss without modifying the server-side model, making it highly practical and particularly suitable for autoregressive text generation tasks.

[0054] In this embodiment, the collaborative reasoning module performs the following forward propagation steps: During the forward propagation process, the client CL first uses the Word2Vector module to perform word segmentation and vectorization on the batch text sequence to generate batch input. ,in, The length of the text represented by a fixed-length sequence of token IDs. Indicates the batch size of the input text; Then, the client CL will input via encryptor model Compared with shallow pre-sub-model Processing to generate encrypted intermediate representations With the original The client CL transmits through an encrypted channel and Securely transmit to cloud server CS.

[0055] After receiving the data, the cloud server CS uses an intermediate sub-model. (Deep Transformer decoder layer, parameter frozen) Process these two sets of intermediate representations separately to obtain the inference output: Intermediate output of the encrypted path: ; Intermediate output of the original path: ; And and Safely return to the client CL; Finally, the client CL uses the decryptor model respectively. With shallow sub-model right and The process is then performed to generate the final collaborative reasoning output: Encrypted path output logits: ; Original path output logits: ; in, Represents the pre-trained model The size of the vocabulary.

[0056] In this embodiment, the accuracy constraints of the dual-layer constraint adversarial fine-tuning module include: To ensure collaborative reasoning output Compared with the original model output Close approximation, using Kullback-Leibler (KL) divergence to quantify collaborative inference output. Compared with the original pre-trained model output The difference in probability distributions between them defines the precision-preserving loss function. Its mathematical expression is:

[0057] in, This represents the softmax function, used to convert the model output into a probability distribution; Temperature is a commonly used temperature parameter in knowledge distillation, used to smooth the probability distribution, multiplied by This can counteract the effect of temperature on gradient scaling, thereby stabilizing the training process; Indicates the batch size of the input text; Represents the pre-trained model The size of the vocabulary; N S The length of the token sequence is fixed. and The original model and the collaborative reasoning model were respectively in the 1st... b The first batch, the first s The token position, the first v The output logits at each vocabulary index; The function of temperature parameters: Through temperature parameters T Smooth the probability distribution to avoid gradient instability caused by overly sharp model output; Gradient stabilization mechanism: multiplying the loss function by T 2 This term is used to offset the effect of temperature parameters on gradient scaling, ensuring the stability of the training process; Constraint objective: To preserve the loss function by minimizing accuracy. Ensure collaborative reasoning output y ′ with the original pre-trained model output y Close approximation maintains the model's inference utility while protecting privacy.

[0058] Furthermore, since the intermediate sub-model deployed on the cloud server CS remains frozen during fine-tuning, relying solely on the accuracy-preserving loss function... To optimize the encryptor and decryptor, the output distribution of the encryptor may still be similar to the original sub-model. Maintaining high consistency is crucial. In this scenario, the encryptor is functionally identical to the previous sub-model, allowing an adversary to perform a high-precision reconstruction attack as if facing the original model. To address this issue, additional privacy constraints are introduced, explicitly inducing a distribution deviation in the encryptor's output, thereby significantly reducing the effectiveness of reconstruction attacks.

[0059] In this embodiment, the privacy constraint mechanism of the two-layer constraint adversarial fine-tuning module includes: To rigorously quantify the difference between the encryptor model output and the previous sub-model output, a privacy loss function based on cosine similarity is adopted. Quantization Encryptor Model intermediate representation of output Compared with shallow pre-sub-model intermediate representation of output The mathematical expression for the distribution difference between them is:

[0060] in, and The cipher model and the original pre-sub-model are respectively in the 1st... b The first batch, the first s The intermediate representation vector at each token position Represents the dot product of vectors, ∥ ||2 represents the L2 norm of the vector; To simultaneously satisfy accuracy and privacy constraints, the decryptor model is jointly optimized to compensate for the distribution bias introduced by the encryptor model. Therefore, the goal of AdvFT is to minimize the inference accuracy while preserving the loss function. At the same time, maximize the loss of privacy The degree of difference in representation being measured.

[0061] To achieve this goal, the accuracy-preserving loss function is used. With privacy loss function Merge into one with adjustable parameters joint loss function Its expression is:

[0062] in, Adjustable parameters to balance accuracy constraints and privacy constraints, It is a small constant used to prevent gradient explosion when the privacy loss is minimal in the early stages of optimization.

[0063] Furthermore, the logarithmic transformation in the second term can rapidly increase the privacy loss in the early stages of optimization, causing the model to quickly deviate from the original representation. As fine-tuning progresses and the difference in representation increases, the logarithmic function naturally slows down the growth rate of the privacy gradient, allowing the optimization process to gradually shift towards maintaining inference accuracy. This adaptive mechanism promotes a more stable and effective trade-off between utility and privacy throughout the fine-tuning process.

[0064] In this embodiment, the backpropagation and parameter update process of the two-layer constraint adversarial fine-tuning module includes: To jointly optimize the encryptor and decryptor models while simultaneously satisfying accuracy and privacy constraints, a gradient-based backpropagation method is employed, along with a joint loss function. (See formula (5)) as the optimization objective, and jointly optimize the client's encryptor model under the conditions of satisfying accuracy constraints and privacy constraints. and decryptor model ; In each iteration, the client determines the loss function based on the joint loss function. Calculate the gradients of the parameters of the two models. Due to the collaborative inference output... Only participate in the accuracy-preserving loss function First calculate The corresponding gradient, and through the decryptor model Backpropagation is performed to obtain the parameter gradients of the decryptor. , and intermediate characterization gradient These gradients are calculated using PyTorch's automatic differentiation mechanism.

[0065] It is important to emphasize the middleware model of cloud servers. The parameters are kept frozen during fine-tuning. Therefore, there is no need to change the gradient. The data is transmitted back to the cloud server, and the client can directly compute its intermediate representation of the previous stage locally. joint gradient :

[0066] in, For accuracy loss gradient, For privacy loss gradient, Adjustable parameters to balance accuracy and privacy, >0 is a small constant to prevent gradient explosion; Joint gradient Through the cryptographer model By performing backpropagation, the parameter gradients of the encryptor can be calculated. and ; Finally, the learning rate is used. The model parameters are updated, as expressed by the following formula: Decryptor model parameter update:

[0067] Encryptor model parameter update:

[0068] Repeat the gradient calculation and parameter update steps above until the model converges or meets the preset stopping criterion, ensuring that the fine-tuned model achieves high inference utility while maintaining the privacy of intermediate representations.

[0069] In this embodiment, after completing the Adversarial Fine-Tuning (AdvFT) process, the client CL removes the original shallow pre-sub-model. With shallow sub-model Only the finely tuned encryptor model is retained. and decryptor model It performs collaborative reasoning together with the cloud server CS.

[0070] First, the client-side CL uses a word embedding encoding module to convert the input text sequence into an input embedding. and will Input to the encryptor model Generate intermediate representations ; Subsequently, the client CL transmitted through a secure channel Send to cloud server CS; Cloud server CS received Then, the intermediate sub-model is frozen using parameters. right Processing is performed to obtain intermediate representations. Then it is sent back to the client CL; Finally, the client CL will Input to the decryptor model Together with the word embedding decoding module, a prediction token is generated and appended to the end of the current input sequence as the new input for the next round of autoregressive prediction; Repeat the above steps of input embedding and encryption, intermediate representation transmission, intermediate sub-model processing, token generation and autoregressive iteration until the preset stopping condition is met, and finally output the generated complete text sequence.

[0071] In summary, this invention effectively protects user privacy during online LLM inference and effectively defends against data reconstruction attacks. While protecting user privacy, it ensures the efficiency of inference and minimizes the loss of inference utility. Furthermore, this invention supports various LLM inference tasks, such as text generation and text classification, and has broad application prospects.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A privacy-preserving adversarial fine-tuning-based LLM collaborative reasoning system, characterized in that, include: The model splitting and deployment module is used to split a pre-trained large language model into shallow pre-sub-models and shallow post-sub-models deployed on the client, and deep intermediate sub-models hosted on the cloud server; wherein, the client and the cloud server achieve collaborative reasoning by exchanging processed intermediate representations. The two-layer constraint adversarial fine-tuning module is embedded in the shallow pre-sub-model and shallow post-sub-model of the client. It includes a customized adapter module for modifying the structure and parameters of the client model under specified conditions to resist potential reconstruction attacks. The adversarial fine-tuning module introduces an auxiliary privacy constraint mechanism to maximize the distribution difference between the encryptor model output and the original pre-sub-model output, while combining it with precision constraints to ensure that the model inference efficiency is maintained while protecting privacy. The collaborative reasoning model, after completing the adversarial fine-tuning process, enables the client and cloud server to perform collaborative reasoning. The collaborative reasoning model includes a client encryptor model, a client decryptor model, and a deep intermediate sub-model, all processed by a two-layer constraint adversarial fine-tuning module.

2. The system according to claim 1, characterized in that: In the model splitting and deployment module, the cloud server CS holds a large language model. Including word embedding encoding module, A Transformer decoder layer and a word embedding decoding module are used to provide inference services to the client CL; The cloud server CS divides the model into three segments according to the hierarchical structure: the shallow sub-model. Intermediate sub-model With shallow sub-model ,in Pre-split points are specified for the client CL and communicated to the cloud server CS; Among them, the shallow pre-sub-model Including word embedding encoding module and front One Transformer decoder layer; intermediate sub-model Including the To the Layered Transformer decoder; shallow sub-model Including the To the The Transformer decoder for the layer and the word embedding decoding module.

3. The system according to claim 2, characterized in that, The deployment method and inference process of the model's split deployment modules include: Cloud server CS retains intermediate sub-model and shallow pre-sub-model With shallow sub-model Send it to the client CL for local deployment, including the shallow pre-sub-model. With shallow sub-model The model size is smaller than the intermediate sub-model. ; When performing collaborative reasoning, given the input embedding of the client CL... The collaborative reasoning process can be represented by the following formula: Among them, the client-side CL utilizes a shallow pre-sub model. For input x The process is performed to generate a first intermediate representation, which is then transmitted to the cloud server CS. The cloud server CS utilizes the intermediate sub-model. The first intermediate representation is processed to generate a second intermediate representation, which is then transmitted back to the client CL. The client CL utilizes the shallow post-submodel. The second intermediate representation is processed to obtain the final output. y .

4. The system according to claim 2, characterized in that: Client-side CL introduces a custom adapter module in the shallow pre-submodel and shallow post-submodel on the client side. To implement modifications to the client-side model, where Indicates the first The parameters of each adapter module, each adapter module adopts a bottleneck neural network structure, consisting of dimensions... Input and output layers Each dimension is (and It consists of a hidden layer and residual connections connecting the input layer and the output of the hidden layer; The shallow pre-submodel will be deployed on the client. After embedding the adapter module, it is defined as an encryptor model. The shallow post-submodel will be deployed on the client. After embedding the adapter module, it is defined as a decryptor model. ,in, This represents the set of parameters for all adapter modules; After the adapter embedding is completed, the model inference formula for the collaborative inference process is updated as follows: in, For the encryption model to input to the client x The processing results This is an intermediate sub-model for cloud servers. ( ) represents the decoding result output by the decryptor model to the intermediate sub-model.

5. The system according to claim 2, characterized in that: In the two-layer constraint adversarial fine-tuning module, the client sub-model and They were respectively converted into encryption model and decryptor model ; The two-layer constraint adversarial fine-tuning module employs a two-layer constraint adversarial fine-tuning strategy, including: Client-side CL and cloud server CS in auxiliary dataset Collaborative cryptographic model and decryptor model Adjust the parameters; During the fine-tuning process, the intermediate sub-model deployed on the cloud server CS The parameters remain frozen and do not participate in parameter updates; During fine-tuning, both accuracy and privacy constraints are simultaneously met: Precision constraints are used to quantify the difference between the fine-tuned collaborative inference output and the original pre-trained model output. Privacy constraints are used to quantify the distributional differences between the output of the encryptor model and the original output of the former sub-model.

6. The system according to claim 2, characterized in that, The collaborative reasoning module performs the following forward propagation steps: During the forward propagation process, the client CL first uses the Word2Vector module to perform word segmentation and vectorization on the batch text sequence to generate batch input. ,in, The length of the text represented by a fixed-length sequence of token IDs. Indicates the batch size of the input text; Then, the client CL will input via encryptor model Compared with shallow pre-sub-model Processing to generate encrypted intermediate representations With the original The client CL transmits through an encrypted channel and Securely transmitted to the cloud server CS; After receiving the data, the cloud server CS uses an intermediate sub-model. Process these two sets of intermediate representations separately to obtain the inference output: Intermediate output of the encrypted path: ; Intermediate output of the original path: ; And and Safely return to the client CL; Finally, the client CL uses the decryptor model respectively. With shallow sub-model right and The process is performed to generate the final collaborative reasoning output: Encrypted path output logits: ; Original path output logits: ; in, Represents the pre-trained model The size of the vocabulary.

7. The system according to claim 5, characterized in that, The accuracy constraints for the two-layer constraint adversarial fine-tuning module include: Kullback-Leibler divergence is used to quantify the collaborative inference output. Compared with the original pre-trained model output The difference in probability distributions between them defines the precision-preserving loss function. Its mathematical expression is: in, This represents the softmax function, used to convert the model output into a probability distribution; Temperature is a commonly used temperature parameter in knowledge distillation, used to smooth the probability distribution, multiplied by This can offset the effect of temperature on gradient scaling; Indicates the batch size of the input text; Represents the pre-trained model The size of the vocabulary; N S The length of the token sequence is fixed. and The original model and the collaborative reasoning model were respectively in the 1st... b The first batch, the first s The token position, the first v The output logits at each vocabulary index.

8. The system according to claim 7, characterized in that, The privacy constraint mechanism of the two-layer constraint adversarial fine-tuning module includes: A privacy loss function based on cosine similarity is used. Quantization Encryptor Model intermediate representation of output Compared with shallow pre-sub-model intermediate representation of output The mathematical expression for the distribution difference between them is: in, and The cipher model and the original pre-sub-model are respectively in the 1st... b The first batch, the first s The intermediate representation vector at each token position. Represents the dot product of vectors, ∥ ||2 represents the L2 norm of the vector; Preserving accuracy loss function With privacy loss function Merge into one with adjustable parameters joint loss function Its expression is: in, Adjustable parameters to balance accuracy constraints and privacy constraints, It is a small constant used to prevent gradient explosion when the privacy loss is minimal in the early stages of optimization.

9. The system according to claim 8, characterized in that, The backpropagation and parameter update process of the two-layer constraint adversarial fine-tuning module includes: A gradient-based backpropagation method is employed, along with a joint loss function. As an optimization objective, the client's encryptor model is jointly optimized while satisfying accuracy and privacy constraints. and decryptor model ; In each iteration, the client determines the loss function based on the joint loss function. Calculate the gradients of the parameters of the two models, due to the output of co-inference. Only participate in the accuracy-preserving loss function First calculate The corresponding gradient, and through the decryptor model Backpropagation is performed to obtain the parameter gradients of the decryptor. , and intermediate characterization gradient ; The client computes its intermediate representation of the previous stage directly on its local machine. joint gradient : in, For accuracy loss gradient, For privacy loss gradient, Adjustable parameters to balance accuracy and privacy, >0 is a small constant to prevent gradient explosion; Joint gradient Through the cryptographer model By performing backpropagation, the parameter gradients of the encryptor can be calculated. and ; Finally, the learning rate is used. The model parameters are updated, as expressed by the following formula: Decryptor model parameter update: Encryptor model parameter update: 。 10. The system according to any one of claims 2 to 9, characterized in that: After completing the adversarial fine-tuning process, the client-side CL removes the original shallow pre-sub-model. With shallow sub-model Only the finely tuned encryptor model is retained. and decryptor model It performs collaborative reasoning together with the cloud server CS; First, the client-side CL uses a word embedding encoding module to convert the input text sequence into an input embedding. and will Input to the encryptor model Generate intermediate representations ; Subsequently, the client CL transmitted through a secure channel Send to cloud server CS; Cloud server CS received Then, the intermediate sub-model is frozen using parameters. right Processing is performed to obtain intermediate representations. Then it is sent back to the client CL; Finally, the client CL will Input to the decryptor model Together with the word embedding decoding module, a prediction token is generated and appended to the end of the current input sequence as the new input for the next round of autoregressive prediction; Repeat the steps of input embedding and encryption, intermediate representation transmission, intermediate sub-model processing, token generation and autoregressive iteration until the preset stopping condition is met, and finally output the generated complete text sequence.

Images