Privacy large model adaptive adjustment method and system based on size model and LoRA

Abstract

The present application relates to a kind of privacy big model self-adapting adjustment method and system based on size model and LoRA, specifically, server side utilizes data generator, and combining differential privacy technology, based on public data set, generate synthetic data;The data distribution of adjacent two times generated on server is calculated KL divergence, if the data distribution KL divergence change is less than divergence threshold value, then the generated data is input to user side model, and user sends output representation to server;If the representation change of adjacent two times is greater than representation threshold value, each user calculates the model prototype corresponding to each category of data owned by it, after server receives the model prototype sent by user, it is projected on the base space that existing LoRA is made up of, if projection residual is greater than or equal to projection threshold value, then add and train LoRA module.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to a privacy-adaptive adjustment method and system for large models based on size models and LoRA. Background Technology

[0002] In federated learning, effectively improving the model's adaptability to diverse user data while protecting user privacy is a crucial research topic. With the increasing heterogeneity and dynamism of user data, achieving efficient collaborative training of models without compromising user privacy has become a key bottleneck in the development of federated learning. Existing research largely focuses on improving model performance through model structure adjustments and privacy protection mechanisms; however, these methods still face many challenges in practical applications, especially in large-scale heterogeneous task scenarios. Balancing privacy protection and performance optimization remains a pressing issue.

[0003] Currently, a common approach is to introduce LoRA technology on the large model side to dynamically adjust model parameters to adapt to the data distribution of different users. However, when dealing with large-scale user data, this method requires adding LoRA modules commensurate with the number of users, facing the dual challenges of high computational complexity and insufficient privacy protection. Furthermore, although differential privacy technology is widely used to protect user data privacy, ensuring that model performance is not affected while protecting privacy remains a pressing issue in practical applications. Therefore, achieving a balance between privacy protection and model performance optimization in large-scale heterogeneous task scenarios is a key technical problem in current federated learning research, with significant theoretical and practical implications. Summary of the Invention

[0004] To address the shortcomings of existing federated learning models in adapting to dynamic data distributions and protecting user data privacy, in a first aspect of this invention, a privacy-adaptive large model adjustment method based on size model and LoRA is provided, the method comprising the following steps:

[0005] The server-side utilizes a data generator and combines it with differential privacy technology to generate synthetic data based on public datasets.

[0006] Calculate the KL divergence for two consecutive data distributions generated on the server. If the change in the KL divergence of the data distribution is less than the divergence threshold, then the generated data is input into the user-side model, and the user sends the output representation to the server.

[0007] If the change in representation between two consecutive iterations exceeds the representation threshold, each user calculates the model prototype corresponding to each category of their data. After receiving the model prototype sent by the user, the server projects it onto the existing base space composed of LoRA. If the projection residual is greater than or equal to the projection threshold, then a new LoRA module is added and trained.

[0008] Preferably, the formula for the projection is:

[0009]

[0010] Where, μ c This is the model prototype, where B is a LoRA on the server side. It is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the user-side small model.

[0011] Preferably, the addition and training of the LoRA module specifically involves:

[0012] Add a new LoRA module L to the large model side new The module is randomly initialized, the parameters of the large model and existing LoRA modules are frozen, and only the newly added LoRA module is trained; the training process uses gradient descent and knowledge distillation loss. To optimize the newly added LoRA module, the updated formula is as follows:

[0013]

[0014] in It is the knowledge distillation loss, used to measure the loss of new LoRA modules. new With user-side module S i Consistency These are the parameters for the newly added LoRA module, where η is the learning rate.

[0015] Preferably, the method further includes:

[0016] If the projection residual is less than the projection threshold, the server will select the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the user's new knowledge; where N is a positive integer.

[0017] Preferably, after adding and training the LoRA module, the method further includes:

[0018] The knowledge added by the heterogeneous small model on the user side is migrated to the newly added LoRA module on the large model side.

[0019] The training of the small model is guided by general knowledge from the large model side through reverse knowledge distillation.

[0020] In a second aspect of the invention, a privacy-adaptive large model adjustment system based on size model and LoRA is provided, the system comprising the following modules:

[0021] The data generation module utilizes a data generator on the server side, combined with differential privacy technology, to generate synthetic data based on public datasets.

[0022] The divergence calculation module calculates the KL divergence for two consecutive data distributions generated on the server. If the change in the KL divergence of the data distribution is less than the divergence threshold, the generated data is input into the user-side model, and the user sends the output representation to the server.

[0023] In the LoRA management module, if the change in representation between two consecutive iterations exceeds the representation threshold, each user calculates the model prototype corresponding to each category of their data. After receiving the model prototype sent by the user, the server projects it onto the base space composed of existing LoRA data. If the projection residual is greater than or equal to the projection threshold, then a new LoRA module is added and trained.

[0024] Preferably, the formula for the projection is:

[0025]

[0026] Where, μ c This is the model prototype, where B is a LoRA on the server side. It is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the user-side small model.

[0027] Preferably, the addition and training of the LoRA module specifically involves:

[0028] Add a new LoRA module L to the large model side new The module is randomly initialized, the parameters of the large model and existing LoRA modules are frozen, and only the newly added LoRA module is trained; the training process uses gradient descent and knowledge distillation loss. To optimize the newly added LoRA module, the updated formula is as follows:

[0029]

[0030] in It is the knowledge distillation loss, used to measure the loss of new LoRA modules. new With user-side module S i Consistency These are the parameters for the newly added LoRA module, where η is the learning rate.

[0031] Preferably, the LoRA module is further configured to:

[0032] If the projection residual is less than the projection threshold, the server will select the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the user's new knowledge; where N is a positive integer.

[0033] Preferably, after adding and training the LoRA module, the method further includes:

[0034] The knowledge added by the heterogeneous small model on the user side is migrated to the newly added LoRA module on the large model side.

[0035] The training of the small model is guided by general knowledge from the large model side through reverse knowledge distillation.

[0036] This invention adds multiple LoRA adapters to a large model structure on the server side, dynamically adjusting model parameters to adapt to changes in data distribution among different users, and prevents user privacy leakage through a localized privacy protection mechanism. Specifically, this invention sets significance thresholds to determine whether user data drift and model structure changes are significant, dynamically adjusting LoRA adapters or adding modules only when significant changes occur, thereby significantly reducing the computational overhead of adaptive model adjustments. Furthermore, during the training of the LoRA modules, an efficient training strategy based on bidirectional knowledge distillation is designed to reduce training complexity. This invention significantly improves the adaptability and computational efficiency of large models in federated learning scenarios without compromising model performance or user privacy protection. Attached Figure Description

[0037] Figure 1 This is a flowchart of Example 1;

[0038] Figure 2 This is a flowchart illustrating the privacy-adaptive adjustment technique for large-scale models based on federated large-scale model collaboration and combination LoRA. Detailed Implementation

[0039] In the embodiments of this invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0040] It is understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0041] In this invention, unless otherwise specified, the same or similar parts between the various embodiments can be referred to each other. In the various embodiments of this invention, and in the various implementation methods / methods / implementations within each embodiment, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the various implementation methods / methods / implementations within each embodiment are consistent and can be mutually referenced. The technical features in different embodiments and the various implementation methods / methods / implementations within each embodiment can be combined according to their inherent logical relationships to form new embodiments, implementation methods, methods, or implementation approaches. The embodiments described below do not constitute a limitation on the scope of protection of this application.

[0042] Example 1

[0043] like Figure 1 The privacy-adaptive adjustment method for large models based on size model and LoRA, as shown, includes the following steps:

[0044] S1, the server side uses a data generator and combines differential privacy technology to generate synthetic data based on public datasets;

[0045] Obtain public datasets from government open data platforms, research institutions, and publicly available datasets published by academia; then select and configure a data generator, preferably a generator from GANs, but also such as variational autoencoders or rule-based generators.

[0046] In one embodiment, the generator is trained using differential privacy stochastic gradient descent (DP-SGD) through gradient clipping and adding noise. The trained data generator is used to generate new synthetic data. The synthetic data is statistically similar to the original public dataset, but does not reveal the privacy information of individuals in the original data.

[0047] S2, calculate the KL divergence for two consecutive data distributions generated on the server. If the change in the KL divergence of the data distribution is less than the divergence threshold, then the generated data is input into the user-side model, and the user sends the output representation to the server.

[0048] The server continuously generates data, and the synthesized data from consecutive generation batches constitutes the data from two adjacent generation batches. The KL divergence between these two batches of data distributions is calculated; the smaller the KL divergence, the more similar the two distributions are. The server compares the KL divergence calculated in this batch with that calculated in the previous batch. If the difference between the two KL divergences is less than a pre-set divergence threshold, the data distributions from the two adjacent generation batches are considered to have little change. The generated synthesized data is then sent to the user-side model. Upon receiving the data, the user-side model processes it and generates a representation, which includes, but is not limited to, the output of the user-side model and the feature vectors extracted by the user-side model. The user-side model then sends the representation back to the server.

[0049] The user-side model is continuously trained with user data. When the structure of the user-side model changes or the user-side data itself undergoes significant changes, the user-side model will also change significantly in response to the data generated by the server. For example, at time t1, the first batch of data generated in two consecutive data sessions is input into the user-side model to obtain representation 1. After adjusting the structure of the user-side model or further training the user-side model using the user's own data, at time t2, the second batch of data generated in two consecutive data sessions is input into the user-side model to obtain representation 2. Representation 1 and representation 2 can be used to determine whether the structure of the user-side model or the user-side training data has undergone significant changes.

[0050] S3, if the change in representation between two consecutive iterations exceeds the representation threshold, each user calculates the model prototype corresponding to each category of their data. After receiving the model prototype sent by the user, the server projects it onto the existing LoRA-based base space. If the projection residual is greater than or equal to the projection threshold, then a new LoRA module is added and trained.

[0051] If the generated data has a KL divergence less than the threshold, but the change in representation between two consecutive data points is greater than the representation threshold, it indicates that either the user-side small model structure has changed, or the user's own data has changed significantly. In this case, the LoRA of the user-side small model and the LoRA of the server side may no longer be able to adapt to the new small model structure or the user's own data.

[0052] When a significant change in representation is detected, the user-side model determines a model prototype for each category based on its locally available data; for example, in a classification model, each category corresponds to a model prototype. The server collects model prototypes for different categories from various users. The server maintains a set of pre-trained LoRA modules. Each LoRA module is fine-tuned by adding a low-rank matrix to learn specific data patterns or tasks. These LoRA modules constitute a base space. The server projects the received user model prototypes onto this existing LoRA base space, representing the new prototype using a combination of existing LoRA modules.

[0053] Projection residuals refer to the difference between the original model prototype and its projection onto the LoRA base space. If this difference is large, exceeding a preset projection threshold, it indicates that the existing LoRA module cannot adequately represent the category information represented by the new model prototype. The server will add a new LoRA module to its model and train this new LoRA module using the received user model prototype or data generated based on that prototype, enabling it to better capture category-specific information in the user data.

[0054] In one embodiment, the formula for the projection is:

[0055]

[0056] Where, μ c This is the model prototype, where B is a LoRA on the server side. It is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the user-side small model.

[0057] Specifically, the user-side model prototype, such as LoRA, has the same structure as multiple LoRAs on the server, but different parameters. The weights of the user-side LoRA and the server's LoRA are obtained. Each user-side LoRA is projected onto a basis composed of the weights of multiple server LoRAs. The least squares method or other methods are used to determine whether the user-side LoRA can be linearly represented by multiple LoRAs on the large model side.

[0058] If multiple LoRA modules on the large model side cannot represent the LoRA module on the user side, then a new LoRA module is added and trained, specifically as follows:

[0059] Add a new LoRA module L to the large model side new The module is randomly initialized, the parameters of the large model and existing LoRA modules are frozen, and only the newly added LoRA module is trained; the training process uses gradient descent and knowledge distillation loss. To optimize the newly added LoRA module, the updated formula is as follows:

[0060]

[0061] in It is the knowledge distillation loss, used to measure the loss of new LoRA modules. new With user-side module S i Consistency These are the parameters for the newly added LoRA module, where η is the learning rate.

[0062] In one embodiment, the method further includes:

[0063] If the projection residual is less than the projection threshold, the server will select the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the user's new knowledge; where N is a positive integer.

[0064] If the projected residual is less than a threshold, the variance of each existing LoRA module needs to be calculated. For example, for the low-rank matrices A and B of the LoRA modules, the variance of all their elements needs to be calculated, or the variance of all LoRA weights needs to be calculated. All existing LoRA modules are sorted according to their variances, and the top N modules with the largest variances are selected. These N selected LoRA modules are marked as new bases to represent the knowledge newly added by the user.

[0065] In an optional embodiment, after adding and training the LoRA module, the method further includes:

[0066] The knowledge added by the heterogeneous small model on the user side is migrated to the newly added LoRA module on the large model side.

[0067] The training of the small model is guided by general knowledge from the large model side through reverse knowledge distillation.

[0068] Example 2

[0069] Step 1: Data Preparation. Artificial data is generated on the server side using the large model-side data generation method in the data security-enhanced time-varying qualitative federated large-scale model collaboration technique. A data generator G is set up on the server side, and differential privacy technology is combined with a public dataset D. pub Generate synthetic data D syn This data is then used for subsequent collaborative training of large and small models. Due to the post-processing characteristics of differential privacy, data privacy can still be protected during subsequent collaborative training.

[0070]

[0071] in, The loss function of the generator is η, where η is the learning rate, and N(0,σ) is the learning rate. 2() is Gaussian noise.

[0072] Step 2: Local Knowledge Change Monitoring. Changes in user-side data, model structure, or task requirements can all lead to alterations in local knowledge. By monitoring user data distribution and model structure, we can determine whether significant changes have occurred in local knowledge.

[0073] 1) Data drift monitoring. Calculate the KL divergence of two consecutive data distributions generated on the server.

[0074]

[0075] If the KL divergence value is greater than the set threshold θ KL This indicates a significant change in user data. This part reflects changes in user-side data through server-side data generation, effectively preventing the leakage of original user data.

[0076] 2) Changes in model structure. If the change in the KL divergence of the data distribution is less than a set threshold θ. KL The generated data will then be input into the user-side model. The user will output the representation φ(S) i The data is sent to the server. If the changes in representations are significant between two consecutive iterations, it indicates a change in the user model structure.

[0077] Step 3: Verify if a new LoRA has been added. When the server detects a significant change in user knowledge, it determines whether the existing LoRA can represent the newly added knowledge on the user's side.

[0078] 1) Each user first needs to calculate the model prototype μ corresponding to each category of the data they possess. c Model prototype μ c It is a centralized representation of user data characteristics, reflecting the distribution center of specific categories of data in the model space. The user will calculate the model prototype μ. c It is sent to the server so that the server can perform subsequent analysis and judgment.

[0079] 2) The server receives the model prototype μ sent by the user. c Then, it is projected onto the existing LoRA base space B. The purpose of the projection is to evaluate whether the existing LoRA base space can cover the feature representation of the user's newly added knowledge. The projection formula is:

[0080]

[0081] Step 4: Dynamically adjust the strategy based on the LoRA module's projection residuals. The specific steps are as follows.

[0082] If the projection residual is less than a specific threshold θ projIf the variance is greater than or equal to the threshold θ, it indicates that the existing base space B is sufficient to cover the user's new knowledge. In this case, no new LoRA module is needed. The server will select several LoRAs with the largest variance from the existing LoRAs as new bases to represent the user's new knowledge. The LoRA modules with the largest variance usually have stronger expressive power and adaptability, and can better capture changes in user data. proj If this is the case, it means that the existing base space B cannot effectively represent the knowledge added by the user. In this case, a new LoRA module is needed to expand the model's representational capabilities and adapt to changes in user-side knowledge.

[0083] Step 5: Train the newly added LoRA module. The specific steps are as follows.

[0084] 1) Add a new LoRA module L to the large model side. new The module was randomly initialized. To ensure training stability and efficiency, the parameters of the large model and existing LoRA modules were frozen, and training was performed only on the newly added LoRA module. Gradient descent was used during training, with knowledge distillation loss. To optimize the newly added LoRA module, making it better suited for user-side small modules S i The knowledge representation is as follows. The updated formula is as follows:

[0085]

[0086] in It is the knowledge distillation loss, used to measure the loss of new LoRA modules. new With user-side module S i Consistency. These are the parameters for the newly added LoRA module. η is the learning rate.

[0087] 2) Transferring knowledge from small model to large model. This involves transferring knowledge from heterogeneous small user models. The newly added knowledge is transferred to the large model side via the newly added LoRA module. new ,based on

[0088]

[0089] in It is the output probability distribution of the small model. This is the output probability distribution of the new LoRA module.

[0090] 3) After transferring knowledge from the small model to the large model, it is also necessary to use the general knowledge from the large model to guide the training of the small model, further improving the performance of the small model. This process is achieved through reverse knowledge distillation, that is, allowing the small model to learn the output distribution of the large model, thereby inheriting the general knowledge of the large model.

[0091] A further improvement of this invention lies in step 2, which proposes a method for monitoring user-side data drift by generating data through a server, effectively protecting user privacy and preventing data leakage. This method proposes dynamic monitoring of local knowledge changes, including data drift and model structure change monitoring, to promptly detect knowledge changes, construct a systematic monitoring process, and introduce quantitative indicators such as KL divergence to improve the accuracy and effectiveness of monitoring.

[0092] A further improvement of this invention is that step 3 proposes a method for verifying whether to add a new LoRA, used to determine whether the existing LoRA can represent the new knowledge added by the user. The user calculates the model prototype corresponding to the data category and sends it to the server. The server then projects the model prototype onto the basis space composed of existing LoRAs. The projection residual is compared with a specific threshold to determine whether a new LoRA needs to be added. If the existing basis can represent the user's new knowledge, several LoRAs with the largest variance are selected as the new basis to represent the user's new knowledge; otherwise, a new LoRA model is added.

[0093] A further improvement of this invention lies in step 5, which proposes a method for knowledge transfer and collaborative training between a large model and a small model. By adding a new LoRA module to the large model and performing targeted training, effective representation and transfer of knowledge from the user-side small module are achieved. Innovatively, knowledge distillation loss is used as the optimization objective. By combining forward knowledge transfer (from small model to large model) and backward knowledge distillation (from large model to small model), the accurate transfer of knowledge from the small model is ensured, while the performance of the small model is improved by utilizing the general knowledge of the large model. During training, the parameters of the large model and the existing LoRA module are frozen, and only the newly added LoRA module is trained, effectively ensuring the stability and efficiency of training. Simultaneously, the newly added LoRA module is optimized using gradient descent to better adapt it to the knowledge representation of the user-side small module.

[0094] Example 3

[0095] Figure 2 This is a flowchart of a privacy-adaptive large model adjustment technique based on federated large-scale model collaboration and combination LoRA. In this embodiment, a federated learning framework based on deep learning is used, where the large model is a pre-trained ResNet-50 model for image classification tasks.

[0096] Step 1: Data Preparation. A data generator is set up on the server side. This generator uses differential privacy technology, employing the CIFAR-10 dataset as a public dataset, to generate synthetic data. For example, the generator adds Gaussian noise (assuming a standard deviation of 0.1) to the original CIFAR-10 image data and combines this with a differential privacy algorithm (such as the Laplacian mechanism, assuming a sensitivity of 1) to generate synthetic image data. This synthetic image data will be used for subsequent co-training of large and small models. Simultaneously, due to the post-processing characteristics of differential privacy, the privacy of the user's original image data can be protected. The generator's loss function is the mean squared error loss (MSE), with a learning rate of 0.001 and a Gaussian noise standard deviation of 0.1. The specific formula can be expressed as follows: Among them, L gen It is the loss function of the generator, N is the number of samples in the synthetic data, and x is the loss function of the generator. i This is a sample of the original CIFAR-10 data. These are the generated synthetic data samples.

[0097] Step 2: Change Detection. Assume there are multiple devices participating in federated learning on the user side, each with its own subset of image data (divided from the CIFAR-10 dataset) and a corresponding model (a small model, such as a simple convolutional neural network CNN).

[0098] 1) Data drift detection. The server calculates the KL divergence of two consecutive generated composite data distributions. For example, if the first generated composite data distribution is P1 and the second is P2, calculate their KL divergence: If the divergence value exceeds a set threshold (assuming the threshold is 0.5), it indicates a significant change in user data. For example, the distribution of image data categories on the user's device may have changed, from being dominated by "airplane" and "car" categories to being dominated by "cat" and "dog" categories. This change will be reflected in the distribution of data generated on the server side, thus effectively preventing the leakage of the user's original image data.

[0099] 2) Model Structure Changes. If the change in the KL divergence of the data distribution is less than a set threshold (assumed to be 0.5), the generated synthetic data is input into the user-side model. For example, the user-side model was originally a simple CNN with 3 convolutional layers and 2 fully connected layers. The user modified the model by adding a convolutional layer. The synthetic data is input into the modified model, and the user sends the output representation (such as feature vectors) to the server. If the change in representation is significant between two consecutive iterations (assumed to be greater than a certain threshold, such as 10, by calculating the Euclidean distance between the representation vectors), it indicates that the user's model structure has changed.

[0100] Step 3: Verify if a new LoRA has been added. Assuming the server detects a significant change in user knowledge, it needs to determine whether the existing LoRA can represent the newly added knowledge from the user's perspective.

[0101] 1) Each user first calculates the model prototype for each category of their data. For example, a user's device has a subset of image data containing three categories: "airplane," "car," and "cat." For the "airplane" category, the user calculates the mean vector of the image data for that category in the model feature space to obtain the model prototype φplane; similarly, the user obtains the model prototype φcar for the "car" category and the model prototype φcat for the "cat" category. The user then sends these model prototypes to the server.

[0102] 2) After receiving the model prototype sent by the user, the server projects it onto the existing LoRA base space B. Assume the existing LoRA base space B consists of 5 LoRA modules, each corresponding to a feature vector. The projection formula can be expressed as: Where, α i It is the projection coefficient, LoRA i This is the feature vector corresponding to the i-th LoRA module in the base space B. The projection residual is used to evaluate whether the existing LoRA base space can cover the feature representation of the user's newly added knowledge.

[0103] Step 4: Dynamic Module Adjustment. If the projection residual is less than a specific threshold (assuming the threshold is 0.1), it indicates that the existing base space B is sufficient to cover the user's new knowledge. For example, if the user adds image data knowledge of the "dog" category, calculating the projection residual reveals that the existing LoRA base space can represent the features of the "dog" category well. In this case, no new LoRA module is needed. The server selects several LoRA modules with the largest variance from the existing LoRA modules as new bases. Assuming the variances of the existing LoRA modules are 0.2, 0.15, 0.1, 0.08, and 0.05, the server selects the top 3 LoRA modules with the largest variances (LoRA modules with variances of 0.2, 0.15, and 0.1) as new bases to represent the user's newly added "dog" category knowledge. This is because LoRA modules with large variances usually have stronger expressive power and adaptability, and can better capture changes in user data. If the projection residual is greater than or equal to the threshold (assuming the threshold is 0.1), it indicates that the existing base space B cannot effectively represent the user's newly added knowledge. For example, if a user adds image data knowledge related to the "truck" category, calculating the projection residual reveals that the existing LoRA base space cannot effectively represent the features of the "truck" category. In this case, a new LoRA module is needed to expand the model's representational capabilities and adapt to changes in user-side knowledge.

[0104] Step 5: Train the new module. The specific steps are as follows:

[0105] 1) Add a new LoRA module to the large model and train it. Add a new LoRA module to the large model (ResNet-50) and randomly initialize it. To ensure training stability and efficiency, freeze the parameters of the large model and the existing LoRA modules, and train only the newly added LoRA module. The training process uses gradient descent and optimizes the new LoRA module with knowledge distillation loss to better adapt it to the knowledge representation of the user-side small modules. Assume the knowledge distillation loss function is: Among them, T i It is the output probability distribution of the user-side small model. This represents the output probability distribution of the newly added LoRA module, where N is the number of samples. The learning rate is set to 0.001, and the update formula can be expressed as: Where θ new These are the parameters for the newly added LoRA module.

[0106] 2) Transferring knowledge from the small model to the large model. New knowledge added to the heterogeneous small model on the user side is transferred to the newly added LoRA module on the large model side. For example, the small model on the user side is trained on image data of the "truck" category and obtains the output probability distribution. This knowledge is transferred to the newly added LoRA module, enabling it to better represent the knowledge of the "truck" category.

[0107] 3) General knowledge from the large model guides the training of the small model. After transferring knowledge from the small model to the large model, it is necessary to use the general knowledge from the large model to guide the training of the small model, further improving its performance. This process is achieved through reverse knowledge distillation, whereby the small model learns the output distribution of the large model, thus inheriting its general knowledge. For example, if the large model (ResNet-50) has good classification performance on the CIFAR-10 dataset, by having the small model learn the output distribution of the large model, it can better inherit the large model's general knowledge of image classification, improving its classification performance for the "truck" category and other categories.

[0108] The above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0109] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, software units executed by a processor, or a combination of both. The software units can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative descriptions of the application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Thus, if such modifications and modifications fall within the scope of the claims and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A privacy large model adaptive adjustment method based on a size model and LoRA, characterized in that, The method comprises the following steps: The server side utilizes a data generator and generates synthetic data based on a public data set in combination with differential privacy technology; The KL divergence of the data distribution generated by the server in adjacent two times is calculated, and if the change of the data distribution KL divergence is less than a divergence threshold, the generated data is input to the user side model, and the user sends the output representation to the server; If the change between two adjacent representations is greater than the representation threshold, each user calculates the model prototype corresponding to each category of data owned by the user, and the server receives the model prototype sent by the user and projects it to the base space composed of existing LoRA If the projection residual is greater than or equal to the projection threshold, a LoRA module is added and trained. The formula of the projection is: wherein, is a model prototype, B is a LoRA on the server side, is a base space composed of all LoRAs on the server side, and the model prototype is a LoRA structure of a small model on the user side; The LoRA module is added and trained, specifically: Add a new LoRA module on the large model side , randomly initialize the module, freeze the parameters of the large model and the existing LoRA module, and only train the new LoRA module; the training process uses gradient descent method to optimize the new LoRA module through knowledge distillation loss , and the update formula is as follows: wherein is a knowledge distillation loss for measuring the consistency of the new LoRA module with the user-side small module , is a parameter of the new LoRA module, is a learning rate; After the LoRA module is added and trained, the following steps are further included: The knowledge added by the user side heterogeneous small model is migrated to the LoRA module added on the large model side; The general knowledge on the large model side is used to guide the training of the small model side in a reverse knowledge distillation manner.

2. The method of claim 1, wherein, The method further comprises: If the projection residual is less than a projection threshold, the server selects N LoRAs with the largest variance from the existing LoRAs as new bases to represent the user added knowledge; wherein N is a positive integer.

3. A privacy large model adaptive adjustment system based on a size model and LoRA, characterized in that, The system comprises the following modules: The data generation module, the server side utilizes a data generator and generates synthetic data based on a public data set in combination with differential privacy technology; The divergence calculation module, the KL divergence of the data distribution generated by the server in adjacent two times is calculated, and if the change of the data distribution KL divergence is less than a divergence threshold, the generated data is input to the user side model, and the user sends the output representation to the server; LoRA management module, if the change of the two adjacent representations is greater than the representation threshold, each user calculates the model prototype corresponding to each category of data owned by the user, and the server receives the model prototype sent by the user and projects it to the base space composed of the existing LoRA If the projection residual is greater than or equal to the projection threshold, a LoRA module is added and trained. The formula of the projection is: wherein, is a model prototype, B is a LoRA on the server side, is a base space composed of all LoRAs on the server side, and the model prototype is a LoRA structure of a small model on the user side; The LoRA module is added and trained, specifically: Adding a new LoRA module on the large model side and randomly initializing the module, freezing the parameters of the large model and the existing LoRA module, and only training the newly added LoRA module; the training process uses gradient descent method to optimize the newly added LoRA module through knowledge distillation loss , and the update formula is as follows: wherein is a knowledge distillation loss for measuring the consistency of the new LoRA module with the user-side small module , is a parameter of the new LoRA module, is a learning rate; After the LoRA module is added and trained, the following steps are further included: The knowledge added by the user side heterogeneous small model is migrated to the LoRA module added on the large model side; The general knowledge on the large model side is used to guide the training of the small model side in a reverse knowledge distillation manner.

Images