Large and small model and LoRA-based privacy large model adaptive adjustment method and system

Abstract

The invention relates to a privacy large model adaptive adjustment method and system based on a large and small model and LoRA, and specifically, a server side generates synthetic data based on a public data set by using a data generator and combining a differential privacy technology; the KL divergence is calculated for data distribution generated two adjacent times on the server, if the change of the KL divergence of the data distribution is smaller than a divergence threshold value, the generated data is input into a user side model, and a user sends an output representation to the server; if two adjacent characterization changes are larger than a characterization threshold value, each user calculates a model prototype corresponding to each category of data owned by the user, after the server receives the model prototype sent by the user, the model prototype is projected to a base space # imgabs0 # composed of existing LoRA, and if a projection residual error is larger than or equal to a projection threshold value, a LoRA module is newly added and trained.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and specifically to a privacy large model adaptive adjustment method and system based on large and small models and LoRA. Background Art

[0002] In federated learning, how to effectively improve the adaptability of the model to diverse user data while protecting user privacy is an important research topic. With the increasing heterogeneity and dynamics of user data, how to achieve efficient collaborative training of the model without revealing user privacy has become a key bottleneck in the development of federated learning. Existing research mainly focuses on how to improve model performance through model structure adjustment and privacy protection mechanisms, but these methods still face many challenges in practical applications. Especially in large-scale heterogeneous task scenarios, how to balance the relationship between privacy protection and performance optimization remains an urgent problem to be solved.

[0003] Currently, a common method is to introduce LoRA technology on the large model side to dynamically adjust model parameters to adapt to different user data distributions. However, when dealing with large-scale user data, this method needs to add LoRA modules equivalent to the number of users, facing the dual challenges of high computational complexity and insufficient privacy protection. In addition, although differential privacy technology is widely used to protect user data privacy, in practical applications, how to ensure that model performance is not affected while protecting privacy remains an urgent problem to be solved. Therefore, how to achieve a balance between privacy protection and model performance optimization in large-scale heterogeneous task scenarios is a key technical issue in current federated learning research, with important theoretical and practical significance. Summary of the Invention

[0004] Aiming at the problems of insufficient adaptability of the large model side to dynamic data distribution and insufficient protection of user data privacy in existing federated learning, in the first aspect of the present invention, a privacy large model adaptive adjustment method based on large and small models and LoRA is provided. The method includes the following steps:

[0005] The server side uses a data generator and, in combination with differential privacy technology, generates synthetic data based on a public dataset.

[0006] Calculate the KL divergence of the data distributions generated on the server twice in succession. If the change in the KL divergence of the data distribution is less than the divergence threshold, the generated data is input into the user-side model, and the user sends the output representation to the server.

[0007] If the change in the representations between two adjacent times is greater than the representation threshold, each user calculates the model prototype corresponding to each category of the data it owns. After the server receives the model prototypes sent by the users, it projects them into the basis space composed of the existing LoRAs. If the projection residual is greater than or equal to the projection threshold, a LoRA module is added and trained.

[0008] Preferably, the projection formula is:

[0009]

[0010] Among them, μ c It is a prototype model, B is a LoRA on the server side, It is the base space composed of all LoRA on the server side, and the model prototype is the LoRA structure of the small model on the user side.

[0011] Preferably, the newly added and trained LoRA module is specifically:

[0012] Add a new LoRA module L on the large model side new , and randomly initialize the module, freeze the parameters of the large model and the existing LoRA module, and only train the newly added LoRA module; the training process uses the gradient descent method, through the knowledge distillation loss To optimize the newly added LoRA module, the update formula is as follows:

[0013]

[0014] in is the knowledge distillation loss, which is used to measure the new LoRA module L new With the user side small module S i consistency, is the parameter of the newly added LoRA module, and η is the learning rate.

[0015] Preferably, the method further comprises:

[0016] If the projection residual is less than the projection threshold, the server will select N LoRAs with the largest variance from the existing LoRAs as the new basis to represent the user's new knowledge; where N is a positive integer.

[0017] Preferably, after the LoRA module is newly added and trained, the method further includes:

[0018] Migrate the newly added knowledge of the heterogeneous small model on the user side to the newly added LoRA module on the large model side.

[0019] Through reverse knowledge distillation, the general knowledge of the large model side is used to guide the training of the small model side.

[0020] In a second aspect of the present invention, a privacy-based large model adaptive adjustment system based on large and small models and LoRA is provided, the system comprising the following modules:

[0021] Data generation module. On the server side, a data generator is used, and combined with differential privacy technology, synthetic data is generated based on a public dataset.

[0022] Divergence calculation module. Calculate the KL divergence of the data distributions generated on the server twice in succession. If the change in the KL divergence of the data distributions is less than the divergence threshold, the generated data is input into the user-side model, and the user sends the output representation to the server.

[0023] LoRA management module. If the change in the representations between two adjacent times is greater than the representation threshold, each user calculates the model prototype corresponding to each category of the data they own. After the server receives the model prototypes sent by the users, it projects them onto the basis space composed of the existing LoRAs If the projection residual is greater than or equal to the projection threshold, a new LoRA module is added and trained.

[0024] Preferably, the formula for the projection is:

[0025]

[0026] where μ c is the model prototype, B is a LoRA on the server side, and is the basis space composed of all LoRAs on the server side. The model prototype is the LoRA structure of the user-side small model.

[0027] Preferably, the adding and training of the LoRA module are specifically as follows:

[0028] Add a new LoRA module L new on the large model side, and perform random initialization on this module. Freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module; the gradient descent method is used in the training process, and the newly added LoRA module is optimized through the knowledge distillation loss The update formula is as follows:

[0029]

[0030] where is the knowledge distillation loss, which is used to measure the consistency between the new LoRA module L new and the user-side small module S i , are the parameters of the newly added LoRA module, and η is the learning rate.

[0031] Preferably, the LoRA module is also used for:

[0032] If the projection residual is less than the projection threshold, the server selects the N LoRAs with the largest variance from the existing LoRAs as the new basis to represent the new knowledge of the user; where N is a positive integer.

[0033] Preferably, after adding and training the LoRA module, the following steps are further included:

[0034] Transfer the knowledge newly added by the heterogeneous small model on the user side to the newly added LoRA module on the large model side.

[0035] Use the general knowledge on the large model side to guide the training of the small model side through the method of reverse knowledge distillation.

[0036] In the present invention, by adding multiple LoRA adapters to the large model structure on the server side, the model parameters are dynamically adjusted to adapt to the changes in the data distribution of different users, and the user privacy leakage is prevented through the local privacy protection mechanism. Specifically, in the present invention, by setting significance judgment thresholds, which are respectively used to determine whether the user data drift and the model structure change are significant, the LoRA adapters or the newly added modules are dynamically adjusted only in the case of significant changes, thereby significantly reducing the computational overhead of model adaptability adjustment. Further, in the training process of the LoRA module, an efficient training strategy based on bidirectional knowledge distillation is designed to reduce the training complexity. Without compromising the model performance and user privacy protection, the present invention significantly improves the adaptability and computational efficiency of the large model in the federated learning scenario. Description of the Drawings

[0037] Figure 1 Is the flowchart of Embodiment 1;

[0038] Figure 2 Is the logical flowchart of the privacy large model adaptive adjustment technology based on the cooperation of federated large and small models and the combination of LoRA. Detailed Embodiments

[0039] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific way for easy understanding.

[0040] It can be understood that the "embodiments" mentioned throughout the specification mean that the specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, the embodiments mentioned throughout the specification do not necessarily refer to the same embodiments. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It can be understood that in various embodiments of the present application, the magnitude of the serial numbers of the various processes does not mean the sequence of execution, and the execution sequence of the various processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0041] In the present invention, unless otherwise specified, the same or similar parts between various embodiments can be referred to each other. In various embodiments of the present invention, as well as in each implementation manner / implementation method / realization method in each embodiment, if there is no special specification and logical conflict, the terms and / or descriptions between different embodiments, as well as between each implementation manner / implementation method / realization method in each embodiment, are consistent and can be cited mutually. The technical features in different embodiments, as well as in each implementation manner / implementation method / realization method in each embodiment, can be combined to form new embodiments, implementation manners, implementation methods, or realization methods according to their internal logical relationships. The following described implementation manners of the present application do not constitute a limitation on the protection scope of the present application.

[0042] Embodiment 1

[0043] As Figure 1 shown, the privacy large model adaptive adjustment method based on the large model and LoRA includes the following steps:

[0044] S1, the server side uses a data generator and combines differential privacy technology to generate synthetic data based on a public data set;

[0045] Obtain a public data set from channels such as the open data platform of government agencies, research institutions, and public data sets released by the academic community; then select and configure a data generator. Preferably, the generator adopts the generator in GAN, and can also adopt, for example, a variational autoencoder, a rule-based generator, etc.

[0046] In one embodiment, the generator is trained in a way of differential privacy stochastic gradient descent (DP-SGD) by means of gradient clipping and adding noise, and the trained data generator is used to generate new synthetic data. The synthetic data is similar to the original public data set in statistical characteristics, but does not disclose the privacy information of individuals in the original data.

[0047] S2, calculate the KL divergence of the data distributions generated on the server twice in succession. If the change in the KL divergence of the data distributions is less than the divergence threshold, input the generated data into the user-side model, and the user sends the output representation to the server.

[0048] The server continuously generates data. The synthetic data generated before and after constitutes the data generated in two adjacent times. Calculate the KL divergence between the distributions of these two batches of data. The smaller the KL divergence, the more similar the two distributions are. The server compares the KL divergence obtained from this calculation with the KL divergence obtained from the previous calculation. If the difference between these two KL divergences is less than a pre-set divergence threshold, it is considered that the distribution of the data generated in two adjacent times does not change significantly. Send the generated batch of synthetic data to the model on the user side. After receiving the data, the model on the user side processes it and generates a representation, which includes but is not limited to the output result of the model on the user side, the feature vector extracted by the model on the user side, etc. The user side sends the representation back to the server.

[0049] The model on the user side is continuously trained with user data. When the structure of the model on the user side changes or the user's own data changes significantly, the change in the data generated by the server for the model on the user side is also relatively large. For example, at time t1, the first batch of data in the data generated in two adjacent times is input into the model on the user side to obtain representation 1. After adjusting the structure of the model on the user side or further training the model on the user side with the user's own data, at time t2, the second batch of data in the data generated in two adjacent times is input into the model on the user side to obtain representation 2. Whether the structure of the model on the user side or the training data on the user side has changed significantly can be judged through representation 1 and representation 2.

[0050] S3, if the change between two adjacent representations is greater than the representation threshold, each user calculates the model prototype corresponding to each category of the data they own. After the server receives the model prototype sent by the user, project it onto the basis space composed of the existing LoRAs If the projection residual is greater than or equal to the projection threshold, add and train the LoRA module.

[0051] The KL divergence of the generated data is less than the threshold, but the change between two adjacent representations is greater than the representation threshold, which means that either the structure of the small model on the user side has changed or the user's own data has changed significantly. At this time, the LoRAs on the user side and the server side may no longer be able to adapt to the new structure of the small model or the user's own data.

[0052] When a large change is detected, the model on the user side will determine a model prototype for each category based on the data it locally has. For example, in a classification model, each category corresponds to a model prototype. The server collects the model prototypes for different categories from each user. The server side maintains a set of pre-trained LoRA modules. Each LoRA module fine-tunes the model by adding a low-rank matrix to learn specific data patterns or tasks. These LoRA modules form a base space. The server projects the received user model prototypes onto this existing LoRA base space and represents the new prototype using a combination of the existing LoRA modules.

[0053] The projection residual refers to the difference between the original model prototype and its projection onto the LoRA base space. If this difference is large and exceeds a preset projection threshold, it means that the existing LoRA modules cannot well represent the category information represented by this new model prototype. The server will add a new LoRA module to its model and use the received user model prototype or data generated based on this prototype to train this new LoRA module so that it can better capture the information of specific categories in the user data.

[0054] In one embodiment, the formula for the projection is:

[0055]

[0056] where μ c is the model prototype, B is a LoRA on the server side, is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the small model on the user side.

[0057] Specifically, the model prototypes on the user side, such as LoRAs, have the same structure as the multiple LoRAs on the server, but different parameters. Obtain the weights of the LoRAs on the user side and the weights of the LoRAs on the server, project each LoRA on the user side onto the base composed of the weights of the multiple LoRAs on the server side, and use methods such as the least squares method to determine whether the LoRAs on the user side can be linearly represented by the multiple LoRAs on the large model side.

[0058] If the multiple LoRAs on the large model side cannot represent the LoRAs on the user side, a new LoRA module will be added and trained. Specifically:

[0059] Add a new LoRA module L new on the large model side and randomly initialize this module. Freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module; the training process uses the gradient descent method and the knowledge distillation loss Optimize the newly added LoRA module, and the updated formula is as follows:

[0060]

[0061] where is the knowledge distillation loss, which is used to measure the consistency between the new LoRA module L new and the small module S on the user side i . are the parameters of the newly added LoRA module, and η is the learning rate.

[0062] In one embodiment, the method further includes:

[0063] If the projection residual is less than the projection threshold, the server will select the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the newly added knowledge of the user; where N is a positive integer.

[0064] If the projection residual is less than the threshold, it is necessary to calculate the variance of each existing LoRA module. For example, for the low-rank matrices A and B of the LoRA module, calculate the variance of all its elements, or calculate the variance of all LoRA weights. Sort all existing LoRA modules according to the variance, select the top N modules with the largest variance, and mark the selected N LoRA modules as new bases to represent the newly added knowledge of the user this time.

[0065] In an alternative embodiment, after adding and training the LoRA module, it further includes:

[0066] Migrate the newly added knowledge of the heterogeneous small model on the user side to the newly added LoRA module on the large model side.

[0067] Use the general knowledge on the large model side to guide the training of the small model side through the method of reverse knowledge distillation.

[0068] Embodiment 2

[0069] Step 1: Data preparation. Generate artificial data on the server side using the data generation method on the large model side in the time-varying heterogeneous federated large and small model collaboration technology with enhanced data security. Set a data generator G on the server side, and combine differential privacy technology to generate synthetic data D pub based on the public dataset D syn for subsequent collaborative training of the large and small models. Due to the post-processing property of differential privacy, data privacy can still be protected in subsequent collaborative training.

[0070]

[0071] where is the loss function of the generator, η is the learning rate, N(0,σ 2) is Gaussian noise.

[0072] Step 2: Local knowledge change monitoring. Changes in user-side data drift, model structure, or task requirements will all lead to changes in local knowledge. By monitoring the user data distribution and model structure, determine whether local knowledge has changed significantly.

[0073] 1) Data drift monitoring. Calculate the KL divergence of the data distributions generated twice in succession on the server.

[0074]

[0075] If the KL divergence value is greater than the set threshold θ KL , it indicates that the user data has changed significantly. This part reflects the changes in user-side data by generating data on the server side, which can effectively prevent the leakage of user's original data.

[0076] 2) Model structure change. If the change in the KL divergence of the data distribution is less than the set threshold θ KL , then input the generated data into the user-side model The user sends the output representation φ(S i ) to the server. If the representations change significantly between two adjacent times, it indicates that the user model structure has changed.

[0077] Step 3: Verify whether a new LoRA is added. When the server detects a significant change in user knowledge, determine whether the existing LoRA can represent the newly added knowledge on the user side.

[0078] 1) Each user first needs to calculate the model prototype μ c corresponding to each category of the data they own. The model prototype μ c is a concentrated representation of the user data features, reflecting the distribution center of specific category data in the model space. The user sends the calculated model prototype μ c to the server for subsequent analysis and judgment.

[0079] 2) After the server receives the model prototype μ c sent by the user, project it onto the basis space B composed of the existing LoRAs. The purpose of the projection is to evaluate whether the existing LoRA basis space can cover the feature representation of the newly added user knowledge. The projection formula is:

[0080]

[0081] Step 4: LoRA module dynamic adjustment strategy based on projection residuals. The specific operations are as follows.

[0082] If the projection residual is less than a specific threshold θ proj, it indicates that the existing base space B is sufficient to cover the user's new knowledge. In this case, there is no need to add a new LoRA module. The server will select several LoRAs with the largest variance from the existing LoRAs as the new base to represent the user's newly added knowledge. LoRA modules with the largest variance usually have stronger expressive power and adaptability, and can better capture the changes in user data. If the projection residual is greater than or equal to the threshold θ proj , it indicates that the existing base space B cannot effectively represent the user's newly added knowledge. At this time, a new LoRA module needs to be added to expand the model's representation ability and adapt to the changes in user-side knowledge.

[0083] Step 5: Train the newly added LoRA module. The specific operations are as follows.

[0084] 1) Add a new LoRA module L on the large model side new , and perform random initialization on this module. To ensure the stability and efficiency of training, freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module. The gradient descent method is used in the training process, and the knowledge distillation loss is used to optimize the newly added LoRA module so that it can better adapt to the knowledge representation of the user-side small module S i . The update formula is as follows:

[0085]

[0086] where is the knowledge distillation loss, which is used to measure the consistency between the new LoRA module L new and the user-side small module S i . are the parameters of the newly added LoRA module. η is the learning rate.

[0087] 2) Transfer of knowledge from the small model side to the large model side. Transfer the newly added knowledge of the user-side heterogeneous small model to the newly added LoRA module L on the large model side new , based on

[0088]

[0089] where is the output probability distribution of the small model, is the output probability distribution of the new LoRA module.

[0090] 3) After completing the transfer of knowledge from the small model side to the large model side, it is also necessary to use the general knowledge on the large model side to guide the training of the small model side to further improve the performance of the small model. This process is achieved through reverse knowledge distillation, that is, letting the small model learn the output distribution of the large model, so as to inherit the general knowledge of the large model.

[0091] A further improvement of the present invention lies in that step 2 proposes a method for monitoring user-side data drift by generating data on the server, effectively protecting user privacy and preventing data leakage. This method monitors changes in local knowledge dynamically, including monitoring data drift and model structure changes, discovers knowledge changes in a timely manner, constructs a systematic monitoring process, and introduces quantitative indicators such as KL divergence to improve the accuracy and effectiveness of monitoring.

[0092] A further improvement of the present invention lies in that step 3 proposes a method for verifying whether a new LoRA is added, which is used to determine whether the existing LoRA can represent the newly added knowledge on the user side. By the user calculating the model prototype corresponding to the data category and sending it to the server, the server then projects the model prototype onto the basis space composed of the existing LoRAs, and uses the comparison between the projection residual and a specific threshold to decide whether a new LoRA needs to be added. If the existing basis can represent the user's new knowledge, then several LoRAs with the largest variance are selected from them as the new basis to represent the user's new knowledge; otherwise, a new LoRA model is added.

[0093] A further improvement of the present invention lies in that step 5 proposes a method for knowledge transfer and collaborative training between the large model and the small model. By adding a new LoRA module on the large model side and conducting targeted training, it realizes the effective representation and transfer of the knowledge of the small module on the user side. Innovatively, the knowledge distillation loss is used as the optimization target, and through the combination of forward knowledge transfer (from the small model to the large model) and reverse knowledge distillation (from the large model to the small model), it not only ensures the accurate transfer of the small model knowledge but also improves the performance of the small model using the general knowledge of the large model. During the training process, the parameters of the large model and the existing LoRA modules are frozen, and only the newly added LoRA module is trained, effectively ensuring the stability and efficiency of the training. At the same time, the newly added LoRA module is optimized by the gradient descent method to better adapt to the knowledge representation of the small module on the user side.

[0094] Embodiment III

[0095] Figure 2 It is a logic flowchart of a privacy large model adaptive adjustment technology based on federated large and small model collaboration and combined LoRA. In this embodiment, a federated learning framework based on deep learning is used, where the large model is a pre-trained ResNet-50 model for image classification tasks.

[0096] Step 1: Data Preparation. Set up a data generator on the server side. It combines differential privacy technology and uses the CIFAR-10 dataset as the common dataset to generate synthetic data. For example, the data generator adds Gaussian noise (assuming the standard deviation of the noise is 0.1) to the original CIFAR-10 image data and combines it with a differential privacy algorithm (such as the Laplace mechanism, assuming its sensitivity is 1) to generate synthetic image data. These synthetic image data will be used for the collaborative training of subsequent large and small models. At the same time, due to the post-processing property of differential privacy, it can protect the privacy of the user's original image data. The loss function of the generator is the mean squared error loss (MSE), the learning rate is 0.001, and the standard deviation of the Gaussian noise is 0.1. The specific formula can be expressed as where L gen is the loss function of the generator, N is the number of samples of the synthetic data, x i is the original CIFAR-10 data sample, is the generated synthetic data sample.

[0097] Step 2: Change Detection. Assume there are multiple devices participating in federated learning on the user side. Each device has its own subset of image data (partitioned from the CIFAR-10 dataset) and a corresponding model (a small model, such as a simple convolutional neural network CNN).

[0098] 1) Data Drift Detection. Calculate the KL divergence of the distributions of the synthetic data generated twice in succession on the server. For example, the distribution of the synthetic data generated for the first time is P1, and the second time is P2. Calculate their KL divergence: If the divergence value is greater than the set threshold (assuming the threshold is 0.5), it indicates that the user data has changed significantly. For example, the class distribution of the image data on the user device has changed, from being mainly "airplane" and "car" classes to being mainly "cat" and "dog" classes. This change will be reflected by the change in the distribution of the data generated on the server side, thus effectively preventing the leakage of the user's original image data.

[0099] 2) Model Structure Change. If the change in the KL divergence of the data distribution is less than the set threshold (assuming it is 0.5), then input the generated synthetic data into the user-side model. For example, the user-side model was originally a simple CNN with 3 convolutional layers and 2 fully connected layers. The user modified the model by adding a convolutional layer. Input the synthetic data into the modified model, and the user sends the output representation (such as a feature vector) to the server. If the change in the representations between two adjacent times is significant (assuming by calculating the Euclidean distance of the representation vectors, the distance is greater than a certain threshold, such as 10), it indicates that the user model structure has changed.

[0100] Step 3: Verify whether to add a LoRA. Assume that the server detects a significant change in the user's knowledge and needs to determine whether the existing LoRA can represent the newly added knowledge on the user side.

[0101] 1) Each user first calculates the model prototype corresponding to each category of the data they own. For example, there is a subset of image data containing three categories of "airplane", "car", and "cat" on the user device. For the "airplane" category, the user obtains the model prototype φplane by calculating the mean vector of the image data of this category in the model feature space; similarly, obtains the "car" category model prototype φcar and the "cat" category model prototype φcat. The user sends these model prototypes to the server.

[0102] 2) After the server receives the model prototypes sent by the users, it projects them onto the basis space B composed of the existing LoRAs. Assume that the existing LoRA basis space B is composed of 5 LoRA modules, and each LoRA module corresponds to a feature vector. The projection formula can be expressed as where α i is the projection coefficient, and LoRA i is the feature vector corresponding to the i-th LoRA module in the basis space B. The projection residual is calculated to evaluate whether the existing LoRA basis space can cover the feature representation of the newly added knowledge of the user.

[0103] Step 4: Module dynamic adjustment. If the projection residual is less than a specific threshold (assume the threshold is 0.1), it means that the existing basis space B is sufficient to cover the new knowledge of the user. For example, the user adds the knowledge of the "dog" category of image data, and it is found by calculating the projection residual that the existing LoRA basis space can better represent the features of the "dog" category. In this case, there is no need to add a LoRA module. The server selects several LoRAs with the largest variance from the existing LoRAs as the new basis. Assume that the variances of the existing LoRA modules are 0.2, 0.15, 0.1, 0.08, 0.05 respectively, and the server selects the top 3 LoRA modules with the largest variance (the LoRA modules with variances of 0.2, 0.15, 0.1) as the new basis to represent the newly added "dog" category knowledge of the user. Because LoRA modules with large variances usually have stronger expressive power and adaptability and can better capture the changes in user data. If the projection residual is greater than or equal to the threshold (assume the threshold is 0.1), it means that the existing basis space B cannot effectively represent the newly added knowledge of the user. For example, the user adds the knowledge of the "truck" category of image data, and it is found by calculating the projection residual that the existing LoRA basis space cannot effectively represent the features of the "truck" category. At this time, a LoRA module needs to be added to expand the representation ability of the model to adapt to the changes in the user-side knowledge.

[0104] Step 5: Train the newly added module. The specific operations are as follows:

[0105] 1) Add a new LoRA module on the large model side and train it. Add a new LoRA module on the large model (ResNet-50) side and randomly initialize this module. To ensure the stability and efficiency of training, freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module. The gradient descent method is used in the training process, and the knowledge distillation loss is used to optimize the newly added LoRA module so that it can better adapt to the knowledge representation of the small module on the user side. Assume the knowledge distillation loss function is: where T i is the output probability distribution of the small model on the user side, is the output probability distribution of the newly added LoRA module, and N is the number of samples. The learning rate is set to 0.001, and the update formula can be expressed as where θ new are the parameters of the newly added LoRA module.

[0106] 2) Transfer the knowledge from the small model side to the large model side. Transfer the newly added knowledge of the heterogeneous small model on the user side to the newly added LoRA module on the large model side. For example, the small model on the user side has been trained on image data of the "truck" category and obtained the output probability distribution. Transfer this knowledge to the newly added LoRA module so that it can better represent the knowledge of the "truck" category.

[0107] 3) Use the general knowledge on the large model side to guide the training of the small model side. After completing the transfer of the knowledge from the small model side to the large model side, it is also necessary to use the general knowledge on the large model side to guide the training of the small model side to further improve the performance of the small model. This process is achieved through reverse knowledge distillation, that is, let the small model learn the output distribution of the large model, so as to inherit the general knowledge of the large model. For example, the large model (ResNet-50) has good classification performance on the CIFAR-10 dataset. By letting the small model learn the output distribution of the large model, the small model can better inherit the general knowledge of the large model for image classification and improve its classification performance for images of the "truck" category and other categories.

[0108] The above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0109] The steps of the methods or algorithms described in the embodiments of the present application may be directly embedded in hardware, a software unit executed by a processor, or a combination of the two. The software unit may be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium may be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Optionally, the storage medium may also be integrated into the processor. The processor and the storage medium may be provided in an ASIC.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0111] Although the present application has been described in connection with specific features and their embodiments, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the present application. Accordingly, the present specification and drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A privacy large model adaptive adjustment method based on a large model and LoRA, characterized in that, The method includes the following steps: On the server side, a data generator is used, and in combination with differential privacy technology, synthetic data is generated based on a public dataset; Calculate the KL divergence of the data distributions generated twice adjacent on the server. If the change in the KL divergence of the data distributions is less than the divergence threshold, the generated data is input into the user-side model, and the user sends the output representation to the server; If the change in representation between two adjacent times is greater than the representation threshold, each user calculates the model prototype corresponding to each category of the data they own. After the server receives the model prototype sent by the user, it projects it onto the basis space composed of the existing LoRAs. If the projection residual is greater than or equal to the projection threshold, a new LoRA module is added and trained.

2. The method according to claim 1, wherein The formula for the projection is: Among them, μ c is the model prototype, and B is a LoRA on the server side. is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the small model on the user side.

3. The method according to claim 1, wherein The specific process of adding and training the LoRA module is as follows: Add a new LoRA module L on the large model side new and randomly initialize this module. Freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module; The training process uses the gradient descent method and optimizes the newly added LoRA module through the knowledge distillation loss The update formula is as follows: Among them is the knowledge distillation loss, which is used to measure the new LoRA module L new and the small module S on the user side i for consistency, are the parameters of the newly added LoRA module, and η is the learning rate.

4. The method according to claim 1, wherein The method further includes: If the projection residual is less than the projection threshold, the server selects the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the newly added knowledge of the user; where N is a positive integer.

5. The method according to claim 1, wherein After adding and training the LoRA module, it further includes: Transfer the newly added knowledge of the heterogeneous small model on the user side to the newly added LoRA module on the large model side; Use the general knowledge on the large model side to guide the training of the small model side through the method of reverse knowledge distillation.

6. A privacy large model adaptive adjustment system based on a large model and LoRA, characterized in that, The system includes the following modules: A data generation module, on the server side, a data generator is used, and in combination with differential privacy technology, synthetic data is generated based on a public dataset; A divergence calculation module, calculate the KL divergence of the data distributions generated twice adjacent on the server. If the change in the KL divergence of the data distributions is less than the divergence threshold, the generated data is input into the user-side model, and the user sends the output representation to the server; LoRA management module. If the change in representation between two adjacent times is greater than the representation threshold, each user calculates the model prototype corresponding to each category of the data they own. After the server receives the model prototype sent by the user, it projects it onto the basis space composed of the existing LoRAs. If the projection residual is greater than or equal to the projection threshold, a new LoRA module is added and trained.

7. The system according to claim 6, wherein The formula for the projection is: Among them, μ c is the model prototype, and B is a LoRA on the server side. is the base space composed of all LoRAs on the server side, and the model prototype is the LoRA structure of the small model on the user side.

8. The system according to claim 6, wherein The specific process of adding and training the LoRA module is as follows: Add a new LoRA module L on the large model side new , and randomly initialize this module. Freeze the parameters of the large model and the existing LoRA modules, and only train the newly added LoRA module; The training process uses the gradient descent method and optimizes the newly added LoRA module through the knowledge distillation loss The update formula is as follows: Among them is the knowledge distillation loss, which is used to measure the new LoRA module L new and the small module S on the user side i for consistency, are the parameters of the newly added LoRA module, and η is the learning rate.

9. The system according to claim 6, wherein The LoRA module is further used for: If the projection residual is less than the projection threshold, the server selects the N LoRAs with the largest variance from the existing LoRAs as new bases to represent the newly added knowledge of the user; where N is a positive integer.

10. The system according to claim 6, wherein After adding and training the LoRA module, it further includes: Transfer the newly added knowledge of the heterogeneous small model on the user side to the newly added LoRA module on the large model side; Use the general knowledge on the large model side to guide the training of the small model side through the method of reverse knowledge distillation.

Images