Model fine-tuning method and risk control method for privacy protection

Abstract

The application discloses a model fine-tuning method for privacy protection, comprising the following steps: embedding an adaptive module into a feature extraction network to obtain an adaptive model; pre-training the adaptive model by using first data, and updating adaptive model parameters except the adaptive module in the pre-training process to obtain a pre-trained model; and distributing the pre-trained model to a user end, so that the user end can train the pre-trained model by using second data, and update parameters of the adaptive module in the training process until the pre-trained model converges, thereby obtaining a user end inference model. The user end inference model can obtain personalized user information, and realize risk control in the personalized learning process. Accordingly, the application also discloses a risk control method for privacy protection.

Description

Technical Field

[0001] This invention relates to a user security management method, and more particularly to a risk control method. Background Technology

[0002] With the popularization and development of technologies such as mobile payment, users not only want a rich user experience, but also want their payment security to be guaranteed. Therefore, in the process of obtaining users' personalized information, the real-time issues of risk control and security algorithm identification are crucial in the current mobile payment and other fields.

[0003] Edge models refer to models deployed on edge nodes, such as users' mobile phones, tablets, and other user devices, used for user-side inference. Compared to cloud models, they are more suitable for solving the aforementioned problems. They not only have higher real-time performance but also fewer model parameters, greatly reducing the link and computational pressure on the server side, enabling lightweight operation. However, lightweight edge models have insufficient generalization ability. Personalized learning can be used to enable edge models to adapt to samples with different distributions. The most common personalized learning method is to fine-tune a pre-trained model using a small number of specific user data samples to achieve personalized representations of users.

[0004] BitFit and LoRa are two commonly used edge model fine-tuning methods. However, BitFit achieves fine-tuning by modifying only the biases without changing the weights of the parameters in the pre-trained model, but at the same time, it loses some of the generalization ability of the pre-trained model, and the model has a small number of parameters, making it difficult to have a good representation search space, and it cannot protect user samples. On the other hand, LoRa fine-tunes by building a bypass encoder-decoder network, which has a better representation search space, but due to insufficient samples, the pre-trained model may not be fully trained or may even overfit, and the performance of the edge model cannot be guaranteed.

[0005] Therefore, we hope to obtain a new model fine-tuning method for privacy protection that balances the number of user samples, the generalization ability of the end model, and the computational cost, while effectively preventing the leakage of user data. Summary of the Invention

[0006] One of the objectives of this invention is to provide a model fine-tuning method for privacy protection. This method can achieve lightweighting of the user-end model, reduce the computational pressure during model training and use, endow the model with strong generalization ability, and reduce the risk of user data leakage during use. This allows the user-end model to safely and efficiently perform personalized learning for users, thereby facilitating the protection of user privacy.

[0007] Based on the above-mentioned objectives, this invention proposes a model fine-tuning method for privacy protection, applied on the server side, comprising:

[0008] An adaptive model is obtained by embedding an adaptive module into a feature extraction network.

[0009] The adaptive model is pre-trained using the first data, and the parameters of the adaptive model other than the adaptive module are updated during the pre-training process to obtain the pre-trained model.

[0010] The pre-trained model is distributed to the user terminal so that the user terminal can train the pre-trained model using the second data and update the parameters of the adaptive module during the training process until the pre-trained model converges to obtain the user terminal inference model.

[0011] This invention, after embedding an adaptive module, updates the parameters of the original feature extraction network (excluding the adaptive module) during the pre-training phase, and updates the parameters of the adaptive module during user-side training, thereby completing the fine-tuning of the end-to-end inference model. This saves a lot of computing resources, ensures the generalization ability of the end-to-end inference model, solves the overfitting problem, and effectively protects user privacy while providing personalized learning.

[0012] Furthermore, the step of embedding the adaptive module into the feature extraction network includes:

[0013] The adaptive module is constructed based on linear or convolutional layers and then embedded into the feature extraction network.

[0014] Furthermore, the step of constructing the adaptive module based on linear or convolutional layers includes:

[0015] A CaSE module is constructed based on convolutional layers to serve as the adaptive module; the CaSE module includes convolutional layers and bypass pooling layers.

[0016] Of course, in other implementations, an adaptive module can also be constructed using a linear layer-based approach.

[0017] Furthermore, the step of embedding the adaptive module into the feature extraction network includes:

[0018] After embedding the adaptive module into the intermediate layer of the feature extraction network.

[0019] Furthermore, the step of embedding the adaptive module into the feature extraction network includes:

[0020] The adaptive module is embedded in a feature extraction network built on the ResNet structure.

[0021] Of course, in other implementations, the adaptive module can also be embedded into other feature extraction network structures in the prior art.

[0022] Furthermore, the step of embedding the adaptive module into a feature extraction network built on the ResNet structure includes:

[0023] The adaptive module is embedded after the activation layer of two adjacent convolutional layers in a feature extraction network built on the ResNet structure.

[0024] Furthermore, the first data includes at least large amounts of data obtained from the server.

[0025] Based on the above-mentioned objectives, this invention proposes a model fine-tuning method for privacy protection, applied to the user end, comprising:

[0026] The server receives the pre-trained model, wherein the pre-trained model is obtained by embedding an adaptive module into a feature extraction network to obtain an adaptive model, and then pre-training the adaptive model using the first data, and updating the adaptive model parameters other than the adaptive module during the pre-training process.

[0027] The pre-trained model is trained using the second data, and the parameters of the adaptive module are updated during the training process until the pre-trained model converges, thus obtaining the user-side inference model.

[0028] Furthermore, the step of training the pre-trained model using the second data and updating the parameters of the adaptive module during the training process includes:

[0029] The pre-trained model is trained using the second data. During the training process, only the parameters of the adaptive module are updated, while the other parameters of the pre-trained model are treated as constants.

[0030] This invention achieves model fine-tuning by freezing parameters in the pre-trained model except for the adaptive module, thereby enabling local training on the user's end, effectively reducing computational overhead, providing users with a customized experience, and improving the accuracy of the end-to-end inference model.

[0031] Furthermore, the step of training the pre-trained model using the second data includes:

[0032] The pre-trained model is trained using the second data for differential privacy.

[0033] In this invention, differential privacy technology is used to effectively protect user data used for personalized learning. Even if the end-inference model is stolen, the user data contained therein will not be leaked. At the same time, it can also resist model inversion attacks and member inference attacks.

[0034] Furthermore, the step of using the second data to perform differential privacy training on the pre-trained model, and updating the parameters of the adaptive module during the training process, includes:

[0035] Solve the sample-by-sample gradient based on the second data;

[0036] The sample-by-sample gradient is processed to obtain the processed gradient;

[0037] The pre-trained model is updated with the processed gradient.

[0038] Furthermore, the step of processing the sample-by-sample gradient includes:

[0039] The sample-by-sample gradient is clipped;

[0040] The cropped gradients are summed sample by sample, and noise is added to obtain the processed gradient.

[0041] Furthermore, the second data is personalized data obtained from the user's terminal.

[0042] Another objective of this invention is to provide a risk control method for protecting user privacy. This method can learn user personal information while ensuring user data security, effectively control risks, and achieve a personalized learning experience that combines low computational cost and high efficiency.

[0043] Based on the above objectives, this invention proposes a risk control method for protecting user privacy, comprising:

[0044] Obtain personalized data from the user's end;

[0045] The user-end personalized data is input into a pre-trained terminal inference model for inference to obtain personalized user information, wherein the terminal inference model is trained using the method described above.

[0046] In fields such as mobile payments, the key to risk control lies in ensuring that while analyzing user needs and providing corresponding services after acquiring user's personalized data, user information leakage is avoided, thereby mitigating risks such as financial losses. In this invention, after inputting user's personalized data, the edge inference model analyzes the data and infers a feature representation of the user's personalized data. This feature representation represents the user's personalized needs and preferences, allowing for the analysis of user needs based on the obtained user personalized information, providing personalized services, and enriching the user experience. Simultaneously, encryption technology is introduced during the training of the edge inference model, protecting user data and corresponding feature representations from leakage. This achieves a balance between enriching the user's personalized experience and protecting user data security, thus realizing risk control.

[0047] Furthermore, the step of inputting the user-end personalized data into a pre-trained terminal inference model for inference includes:

[0048] The user-end personalized data is input into a pre-trained end-to-end inference model that uses the CaSE module as an adaptive module. The CaSE module includes convolutional layers and bypass pooling layers. During inference, constants are used instead of bypass pooling layers.

[0049] Another objective of this invention is to provide a model fine-tuning device for privacy protection. This device enables lightweighting of the user-end model, reducing the computational pressure during model training and use while endowing the model with strong generalization ability. It also reduces the risk of user data leakage during use, allowing the user-end model to safely and efficiently perform personalized learning for users, thereby facilitating the protection of user privacy.

[0050] To achieve the above objectives, the present invention provides a model fine-tuning device for privacy protection, comprising:

[0051] The embedding module embeds the adaptive module into the feature extraction network to obtain the adaptive model;

[0052] The pre-training module uses the first data to pre-train the adaptive model and updates the parameters of the adaptive model other than the adaptive module during the pre-training process to obtain the pre-trained model. Then, the pre-trained model is distributed to the user terminal so that the user terminal can use the second data to train the pre-trained model and update the parameters of the adaptive module during the training process until the pre-trained model converges to obtain the user terminal inference model.

[0053] Furthermore, the embedding module constructs a CaSE module based on convolutional layers as the adaptive module, and embeds the adaptive module into the feature extraction network; the CaSE module includes convolutional layers and bypass pooling layers.

[0054] To achieve the above objectives, the present invention provides a model fine-tuning device for privacy protection, comprising:

[0055] The training module receives the pre-trained model from the server and trains the pre-trained model using the second data. During the training process, the parameters of the adaptive module are updated until the pre-trained model converges, thus obtaining the user-side inference model.

[0056] The pre-trained model is obtained by embedding the adaptive module into the feature extraction network on the server side, then pre-training the adaptive model using the first data, and updating the adaptive model parameters other than the adaptive module during the pre-training process.

[0057] Furthermore, during the training process of the pre-trained model using the second data, the training module only updates the parameters of the adaptive module, treating the other parameters of the pre-trained model as constants.

[0058] Furthermore, the training module uses the second data to perform differential privacy training on the pre-trained model.

[0059] Another objective of this invention is to provide a risk control device for protecting user privacy. This device can learn user personal information while ensuring user data security, effectively control risks, and achieve a personalized learning experience that combines low computational cost and high efficiency.

[0060] To achieve the above objectives, the present invention also provides a risk control device for protecting user privacy, comprising:

[0061] The inference module inputs the acquired user-end personalized data into a pre-trained end-to-end inference model for inference to obtain personalized user information. The end-to-end inference model is trained using any one of the model fine-tuning methods for privacy protection described above.

[0062] Another object of the present invention is to provide a computer-readable storage medium, characterized in that the storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described above in the model fine-tuning method for privacy protection.

[0063] Another object of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the program, it implements the steps of any of the above-described methods for model fine-tuning for privacy protection.

[0064] The model fine-tuning method for privacy protection described in this invention has the following beneficial effects:

[0065] The privacy-preserving model fine-tuning method described in this invention, after embedding an adaptive module, updates parameters outside the adaptive module during the pre-training phase and updates the parameters of the adaptive module during user-side training, thereby completing the fine-tuning of the end-to-end inference model. This saves a significant amount of computational resources, ensures the generalization ability of the end-to-end inference model, solves the overfitting problem, and effectively protects user privacy while providing personalized learning. In addition, it allows users to obtain a customized experience and improves the accuracy of the end-to-end inference model.

[0066] In a preferred embodiment of the present invention, the model fine-tuning method for privacy protection described herein employs differential privacy technology to effectively protect user data used for personalized learning. Even if the end-inference model is stolen, the user data contained therein will not be leaked. At the same time, it can also resist model inversion attacks and member inference attacks.

[0067] The risk control method for protecting user privacy described in this invention has the following beneficial effects:

[0068] After inputting the user's personalized data, the edge inference model analyzes the data and infers the feature representation of the user's personalized data. This feature representation represents the user's personalized needs and preferences. Therefore, based on the obtained user personalized information, the model can analyze user needs and provide corresponding personalized services, thus enriching the user experience. At the same time, the edge inference model introduces encryption technology during training, which can protect user data and corresponding feature representations from being leaked. This achieves a balance between enriching the user's personalized experience and protecting user data security, and realizes risk control. Attached Figure Description

[0069] Figure 1 An exemplary flowchart of one implementation of the model fine-tuning method for privacy protection described in this invention is shown.

[0070] Figure 2 An exemplary diagram of the data flow of the CaSE module in the adaptive module of the present invention is shown in one embodiment.

[0071] Figure 3 An exemplary flowchart of another implementation of the privacy-preserving model fine-tuning method of the present invention is shown.

[0072] Figure 4 An exemplary flowchart of the steps of the privacy-preserving model fine-tuning method of the present invention is shown in a preferred embodiment.

[0073] Figure 5An exemplary flowchart of one embodiment of the risk control method for protecting user privacy according to the present invention is shown.

[0074] Figure 6 An exemplary block diagram of the model fine-tuning device for privacy protection according to the present invention is shown in one embodiment. Detailed Implementation

[0075] The model fine-tuning and risk control method and apparatus for privacy protection described in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. However, this detailed description does not constitute a limitation of this invention.

[0076] In fields such as mobile payments, risk control is of paramount importance. How to ensure that user information is not leaked while providing services based on personalized user information analysis, thereby mitigating risks such as financial loss, has become a major concern in related technical fields. Typically, risk control algorithms operate in environments including the server side and the user side, also known as the cloud side and the edge side. Risk control is implemented through an inference model. After inputting the user's personalized data, the inference model analyzes the data and infers a feature representation of the user's personalized data. This feature representation represents the user's personalized needs and preferences; simultaneously, encryption techniques are used to protect user data and the corresponding feature representation from leakage.

[0077] In this model, the inference model used to infer personalized user information on the cloud side is called a cloud model or cloud inference model, while the model used to infer personalized user information on the edge side is called an edge model or edge inference model. Edge models have an advantage over cloud models in terms of controlling computational pressure. They typically have fewer model parameters and shorter inference times, enabling lightweight operation. Furthermore, since edge models use personalized user data, they can be adjusted in real-time based on updates to this data, thus better ensuring the accuracy of inference and improving efficiency. However, lightweight edge models often suffer from insufficient generalization ability due to the limited number of training samples. Therefore, personalized learning methods are introduced. This involves fine-tuning a pre-trained model using a small number of specific user data samples, allowing the edge model to have different parameters or inference methods for different specific users. This ensures the lightweight edge model's adaptability to samples with different distributions.

[0078] However, model fine-tuning needs to be implemented on the user's end. With limited computing resources, performing full parameter fine-tuning on the entire end-user model incurs significant computational overhead, contradicting the lightweight objective. Furthermore, since fine-tuning uses only a small number of specific user data samples, the end-user model develops a memory of that data. If the model is stolen, this data can easily be leaked, threatening user information security. Moreover, the end-user model only exhibits high performance on that specific data set, and may even overfit, making it unsuitable for a wider sample space.

[0079] Therefore, the solution described in this invention provides a model fine-tuning method and risk control method for privacy protection, which can save a lot of computing resources, ensure the generalization ability of the edge inference model, solve the overfitting problem, and effectively protect user privacy while providing personalized learning; in addition, it can enable users to obtain a customized experience and ensure the accuracy of the edge inference model.

[0080] Figure 1 An exemplary flowchart of one implementation of the model fine-tuning method for privacy protection described in this invention is shown.

[0081] like Figure 1 As shown, in one embodiment of the present invention, a model fine-tuning method for privacy protection is proposed and applied to the server side, comprising:

[0082] 100: Embed the adaptive module into the feature extraction network to obtain the adaptive model.

[0083] Feature extraction networks are one of the most widely used network types in the field of deep learning. Classic feature extraction network models include LeNet, ResNet, and VGG. Faced with various types of input samples such as text, images, and audio, feature extraction networks can extract feature representations of each sample to distinguish different samples, facilitating subsequent similarity comparisons, classifications, and other tasks. They are widely used in various fields such as text comparison and face recognition.

[0084] This invention selects a feature extraction network model from the prior art as the initial model, and performs pre-training and fine-tuning based on the existing model structure and parameters, which can save a lot of time and cost for building and debugging the model.

[0085] In some embodiments, the adaptive module can be embedded into a feature extraction network built on a ResNet architecture.

[0086] Of course, in other embodiments, those skilled in the art can also use other types of feature extraction network models known to them, such as LeNet or VGG, as needed. Adaptive modules typically contain adaptive network blocks or layers, used to embed into existing feature extraction network models for small-sample-based model fine-tuning. The resulting model can more accurately serve specific samples and achieve the target task. Since the adaptive module is relatively independent of the original initial model, during training, only the parameters of the adaptive module can be updated, or only the parameters of the initial model can be updated, or all of the above parameters can be updated simultaneously. This allows for selective updating of model parameters according to different training objectives, reducing computational overhead. The number of adaptive modules can be one or more, determined by those skilled in the art based on the required feature extraction performance and computational capabilities of the adaptive model. By introducing adaptive modules, updating all parameters in the entire adaptive model during each training iteration is effectively avoided, aligning with the fine-tuning concept, reducing computational pressure, and improving the efficiency of adaptive model training.

[0087] Among existing model fine-tuning techniques, the BitFit method only fine-tunes the biases of the parameters. By modifying only the biases without changing the parameters in the pre-trained network, it saves significant time and memory costs, making it suitable for fine-tuning scenarios on edge devices. However, fine-tuning only the biases essentially sacrifices some of the generalization ability of the adaptive model. Furthermore, due to the small number of bias parameters, it's difficult for the adaptive model to have a good representation search space. Embedding adaptive modules for model fine-tuning allows for flexible control over the number of adjustable parameters within the modules, resulting in a larger search space and ensuring the generalization ability of the adaptive model. Additionally, there's the LoRa method, which fine-tunes only the encoding / decoding network and the downstream classifier by building a bypass encoder / decoder network around the initial model. In this method, the initial model only provides the input representation. This method effectively addresses the problem of the small representation search space in the BitFit method. However, it has a larger problem: due to the small number of samples used for fine-tuning, the bypass network cannot be fully trained. Training too many batches on this small sample set can even lead to overfitting, significantly degrading the performance of the adaptive model. Compared to this method, the adaptive module is directly embedded in the initial model instead of being used as a bypass network and trained together with the initial model. This avoids the problem of insufficient training and reduces the possibility of overfitting.

[0088] In some embodiments, the adaptive module is built on linear or convolutional layers and embedded in the feature extraction network.

[0089] In some specific embodiments, a CaSE module is constructed based on convolutional layers as an adaptive module; the CaSE module includes convolutional layers and bypass pooling layers.

[0090] The CaSE (Contextual Squeeze-Excite) module is an adaptive network block consisting of two convolutional layers and a side-channel pooling layer, with a corresponding activation layer following the convolutional layer. Its characteristic is that the data flow is inconsistent between the training and inference phases of the edge inference model. Figure 2 An exemplary diagram of the data flow of the CaSE module in the adaptive module of the present invention is shown in one embodiment.

[0091] like Figure 2 As shown, the bypass pooling layer includes structures such as spatial pooling layer, content pooling layer, and linear layer. When the edge inference model is trained, data flows into each layer, and the weights of the bypass pooling layer are remembered during the process. When the edge inference model performs inference, the entire bypass pooling layer is converted into a constant based on the weights remembered during training, and the data no longer passes through this layer, thereby saving the computational overhead of the edge inference model and accelerating the inference efficiency of the edge inference model.

[0092] Of course, in other embodiments, adaptive modules can also be constructed using a linear layer-based approach.

[0093] In some embodiments, the adaptive module is embedded after the intermediate layer of the feature extraction network.

[0094] In some specific embodiments, the adaptive module can be embedded after the activation layer of two adjacent convolutional layers in a feature extraction network built on the ResNet architecture.

[0095] 110: The first data is used to pre-train the adaptive model, and the parameters of the adaptive model other than the adaptive module are updated during the pre-training process to obtain the pre-trained model.

[0096] In some embodiments, the first data includes at least large data obtained from the server.

[0097] The big data obtained from the server includes various general public datasets. These datasets are characterized by their large quantity and wide coverage, which can ensure the training efficiency of the adaptive model and the feature representation effect of the obtained pre-trained model.

[0098] Using large amounts of data from the server to pre-train the adaptive model is to enhance the adaptability between the adaptive module and the initial model. This allows the adaptive model to retain the feature representation capabilities of the original feature extraction network while adapting to the changes in model structure brought about by embedding the adaptive module. At this point, the model may not be consistent with the target task. It is only necessary to ensure that the pre-trained model has good feature representation capabilities. Therefore, this pre-training stage only updates the parameters of the adaptive model other than the adaptive module. This avoids updating all parameters of the entire model, which is in line with the lightweight concept and reduces the network and computational pressure on the server.

[0099] 120: Distribute the pre-trained model to the user terminal so that the user terminal can train the pre-trained model using the second data and update the parameters of the adaptive module during the training process until the pre-trained model converges, thus obtaining the user terminal inference model.

[0100] In this model, the parameters of the adaptive module are only updated on the user side, while the other parameters of the pre-trained model are frozen as constants. At this time, the training process of the pre-trained model aims to improve the ability of personalized learning, which also avoids updating all the parameters of the entire model and improves training efficiency.

[0101] In some embodiments, the second data is personalized data obtained from the user's device.

[0102] In some embodiments, a second set of data is used to perform differential privacy training on the pre-trained model.

[0103] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the model is stolen, the user data it contains will not be leaked. It also resists model inversion attacks and member inference attacks. The specific steps of differential privacy training will be described later when discussing the user side.

[0104] Figure 3 An exemplary flowchart of another implementation of the privacy-preserving model fine-tuning method of the present invention is shown.

[0105] like Figure 3 As shown, in one embodiment of the present invention, a model fine-tuning method for privacy protection is proposed and applied to the user end, comprising:

[0106] 200: Receive the pre-trained model sent by the server. The pre-trained model is obtained by embedding the adaptive module into the feature extraction network to obtain the adaptive model, and then using the first data to pre-train the adaptive model. During the pre-training process, the parameters of the adaptive model other than the adaptive module are updated.

[0107] This invention selects a feature extraction network model from the prior art as the initial model, and performs pre-training and fine-tuning based on the existing model structure and parameters, which can save a lot of time and cost for building and debugging the model. In some embodiments, the adaptive module can be embedded into the feature extraction network built based on the ResNet structure.

[0108] Of course, in other embodiments, those skilled in the art may also use other types of feature extraction network models known to them, such as LeNet or VGG, as needed.

[0109] Adaptive modules are typically embedded in existing feature extraction network models for small-sample-based model fine-tuning. The resulting model can more accurately serve specific samples and achieve the target task. Because the adaptive module is relatively independent of the original initial model, during training, only the parameters of the adaptive module can be updated, or only the parameters of the initial model can be updated, or all of the above parameters can be updated simultaneously. This allows for selective updating of model parameters according to different training objectives, reducing computational overhead. By introducing adaptive modules, the need to update all parameters in the entire adaptive model during each training iteration is effectively avoided, aligning with the fine-tuning concept, reducing computational pressure, and improving the efficiency of adaptive model training.

[0110] In some specific embodiments, a CaSE module is constructed based on convolutional layers to be embedded in the feature extraction network as an adaptive module; the CaSE module includes convolutional layers and bypass pooling layers.

[0111] The CaSE module is characterized by a different data flow during edge inference model training and inference. During training, data flows into each layer, and the weights of the side-pooling layers are remembered during this process. During inference, the entire side-pooling layer is converted to a constant based on the weights remembered during training, and data no longer passes through this layer. This saves computational overhead during model inference and accelerates the inference efficiency of the edge inference model. Of course, in other embodiments, an adaptive module can also be built based on linear layers.

[0112] In some embodiments, the first data includes at least large data obtained from the server.

[0113] The large datasets acquired from the server include various general public datasets. These datasets are characterized by their large quantity and wide coverage, ensuring the training efficiency of the adaptive model and the feature representation performance of the pre-trained model. Using server-side large datasets for pre-training the adaptive model enhances the adaptability between the adaptive module and the initial model. This allows the adaptive model to retain the feature representation capabilities of the original feature extraction network while adapting to the structural changes brought about by embedding the adaptive module. The pre-trained model obtained at this stage may not necessarily be consistent with the target task; it is only necessary to ensure that the pre-trained model has good feature representation capabilities. Therefore, this pre-training stage only updates the parameters of the adaptive model other than the adaptive module, avoiding updates to all parameters of the entire model. This aligns with the lightweight design principle and reduces the network and computational load on the server.

[0114] 210: The pre-trained model is trained using the second data. During the training process, the parameters of the adaptive module are updated until the pre-trained model converges, thus obtaining the user-side inference model.

[0115] In some embodiments, the second data is personalized data obtained from the user's device.

[0116] The user end includes end devices such as smartphones and smartwatches. These personal devices typically store the device owner's personalized data, such as consumption preferences, geographical location, and family relationships. The pre-trained model, trained using the first set of server-side data, possesses excellent feature representation capabilities. However, compared to the first set of data, the second set of user-specific data covers a much smaller scope, but contains far more precise and richer information. Fine-tuning using this user-specific data helps the pre-trained model quickly learn the preferences of specific users, ensuring the quality of personalized learning.

[0117] In the aforementioned stage, server-side big data was used to pre-train the adaptive model, resulting in a pre-trained model. During the pre-training process, only parameters other than the adaptive module were updated, with the aim of training the feature representation capability of the pre-trained model.

[0118] In some embodiments, the pre-trained model is trained using second data, and only the parameters of the adaptive module are updated during the training process, while the other parameters of the pre-trained model are treated as constants.

[0119] The parameters of the adaptive module are only updated on the user side, while the other parameters of the pre-trained model are frozen as constants after pre-training. At this time, the process of using second data to train the pre-trained model aims to improve the model's ability to learn individually. This not only avoids updating all parameters of the entire model and improves training efficiency, but also enables targeted training of specific samples, thereby enhancing the accuracy of the edge inference model.

[0120] like Figure 4 As shown, preferably, in some embodiments, the pre-trained model is trained with differential privacy using second data.

[0121] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the edge inference model is stolen, the user's personalized data contained within it will not be leaked. It also resists model inversion attacks and membership inference attacks.

[0122] In some specific embodiments, differential privacy training of the pre-trained model is performed using second data, and the step of updating the parameters of the adaptive module during training includes:

[0123] Solve the sample-by-sample gradient based on the second data;

[0124] The gradient is processed sample by sample to obtain the processed gradient;

[0125] The pre-trained model is updated with the processed gradient.

[0126] The second dataset contains both user-specific data and corresponding data labels. After inputting the user-specific data into the pre-trained model, the loss is calculated based on the forward output and the corresponding data labels. The sample-by-sample gradient can then be obtained from the loss.

[0127] To protect users' personalized data, the differential privacy method randomizes the gradient for each sample. Even if the model is stolen, attackers cannot obtain the corresponding user personalized data based on the randomly generated gradient, thereby greatly reducing the risk of user information leakage and providing a relatively secure payment environment.

[0128] In some more specific embodiments, the step of processing the sample-by-sample gradient includes:

[0129] Clipping of gradients per sample;

[0130] The sample-by-sample gradients after cropping are summed, and noise is added to obtain the processed gradient.

[0131] It should be noted that a clipping norm C is used when clipping the sample-by-sample gradients. This means that every gradient with a L2 norm greater than C is scaled down to the size of C. After clipping all sample-by-sample gradients in the current batch, they are summed and noise is added to obtain the randomized gradient. This gradient is used to update the parameters of the pre-trained model, and then the next batch of training is performed until the model converges.

[0132] Optionally, in a more specific embodiment, the pre-trained model is trained using second data with differential privacy, and the steps for updating the parameters of the adaptive module during training are as follows:

[0133] Let ξ represent the sample of user personality data, and let b be the number of samples. Then the sample data can be represented as (ξ1, ξ2, ..., ξ). b The corresponding data label is represented by y, i.e., (y1, y2, ..., y3). b After inputting user personality data into the pre-trained model x, the model is then processed based on the forward output results (f(x, ξ1), f(x, ξ2), ..., f(x, ξ)). b The loss is calculated using the data labels and the corresponding data labels. Based on the loss, the sample-by-sample gradient g can be obtained, i.e., (g) t，1 g t，2 , ..., g t，b ), where t represents the training batch.

[0134] Next, the gradient per sample is clipped according to the clipping norm, as follows:

[0135]

[0136] Among them, g t，i Let be the gradient of the i-th sample in the t-th batch; is the gradient after clipping; Clip() is the clipping function; C is the clipping norm.

[0137] After clipping all sample-by-sample gradients in the current batch, sum them and add noise to obtain the randomized gradient, as follows:

[0138]

[0139] in, The gradient is the result of the processing; N() is the noise function, σ is the attribute parameter in the noise function; b is the number of samples; i is the index of the sample.

[0140] The steps for updating the parameters of a pre-trained model using the processed gradients are as follows:

[0141]

[0142] Where x represents the pre-trained model; η represents the learning rate; x t This represents the model obtained from training the t-th batch.

[0143] After updating all parameters of the adaptive module in the pre-trained model according to the above formula, the next batch of training is carried out until the model converges.

[0144] Figure 4 An exemplary flowchart of the steps of the privacy-preserving model fine-tuning method of the present invention is shown in a preferred embodiment.

[0145] like Figure 4 As shown, in a preferred embodiment, on the cloud side, i.e. the server side, the existing feature extraction network ResNet is selected as the initial model, and the CaSE module is selected as the adaptive module and embedded into the feature extraction network built based on the ResNet structure to obtain an adaptive model. Then, the adaptive model is pre-trained using cloud-side data, and the adaptive model parameters other than the adaptive module are updated during the pre-training process. After obtaining the pre-trained model, the adaptive model parameters other than the adaptive module are frozen, and the trained pre-trained model is sent to the user terminal. The cloud-side data is the first data, which includes at least the big data obtained from the server side.

[0146] On the edge, i.e. the user end, the pre-trained model is received from the cloud side as the edge training model. Differential privacy training is performed on the edge training model using edge data. During the training process, the parameters of the adaptive module are updated until the pre-trained model converges, and the user end inference model is obtained. The edge data is the second data, which is the personalized data obtained from the user end.

[0147] The steps involved in differential privacy training include:

[0148] The edge data includes both user personality data and corresponding edge labels. After inputting the user personality data into the pre-trained model, the loss is calculated based on the forward output and the corresponding data labels, and the sample-by-sample gradient is obtained from the loss. The sample-by-sample gradients are then clipped using a clipping norm C, which means that every gradient with a L2 norm greater than C is scaled down to the size of C. After clipping all sample-by-sample gradients in the current batch, they are summed and noise is added to obtain a randomized gradient. This gradient is used to update the parameters of the pre-trained model, and then the next batch of training is performed until the model converges.

[0149] The privacy-preserving model fine-tuning method described in this invention, after embedding an adaptive module, updates parameters outside the adaptive module on the server side and updates the parameters of the adaptive module during user-side training, thereby completing the fine-tuning of the end-to-end inference model. This saves significant computational resources, ensures the generalization ability of the end-to-end inference model, solves the overfitting problem, and effectively protects user privacy while providing personalized learning. Furthermore, it allows users to obtain a customized experience and improves the accuracy of the end-to-end inference model. In addition, the method described in this invention employs differential privacy technology to effectively protect user data used for personalized learning; even if the end-to-end inference model is stolen, the user data it contains will not be leaked. It also resists model inversion attacks and member inference attacks.

[0150] Figure 5 An exemplary flowchart of one implementation of the risk control method for user-side privacy protection according to the present invention is shown.

[0151] like Figure 5 As shown, in another embodiment of the present invention, a risk control method for protecting user privacy is proposed, which includes:

[0152] 300: Obtain user-specific data.

[0153] The user end includes end devices, such as smartphones, smartwatches, and other electronic devices. Personal data is stored on these personal end devices, typically including the device owner's personal data, such as consumption preferences, geographical location, and family relationships. When an application used by a user connects to the internet and is granted certain data access permissions, the user's habits and preferences while using the application are remembered as personal data and stored on the end device. For example, when a user orders takeout using an application, the application can obtain the user's geographical location and dietary preferences; it can also obtain information about the user's lifestyle from their online shopping activity.

[0154] 310: Input the user's personalized data into the pre-trained terminal inference model for inference to obtain personalized user information. The terminal inference model is trained using any of the model fine-tuning methods mentioned above for privacy protection.

[0155] In fields such as mobile payments, the key to risk control lies in ensuring that while analyzing user needs and providing corresponding services after acquiring user's personalized data, user information leakage is avoided, thereby mitigating risks such as financial losses. After inputting user's personalized data, the edge inference model analyzes the data and infers a feature representation of the user's personalized data. This feature representation represents the user's personalized needs and preferences, allowing for the analysis of user needs based on the obtained user personalized information, providing personalized services, and enriching the user experience. Simultaneously, the edge inference model incorporates encryption technology during training, protecting user data and corresponding feature representations from leakage. This achieves a balance between enriching the user's personalized experience and protecting user data security, thus realizing risk control.

[0156] Preferably, in some embodiments, differential privacy training is employed when training the peer inference model.

[0157] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the edge inference model is stolen, the user's personalized data contained within it will not be leaked. It also resists model inversion attacks and membership inference attacks.

[0158] To protect users' personalized data, the differential privacy method randomizes the gradient for each sample. Even if the edge inference model is stolen, attackers cannot obtain the corresponding user personalized data based on the randomly generated gradient, thereby greatly reducing the risk of user information leakage and providing a relatively secure payment environment.

[0159] In some embodiments, the user-end personalized data is input into a pre-trained end-to-end inference model that uses a CaSE module as an adaptive module. The CaSE module includes convolutional layers and bypass pooling layers. During inference, constants are used instead of bypass pooling layers.

[0160] It should be noted that the CaSE module is an adaptive network block consisting of two convolutional layers and a side-pooling layer. Its characteristic is that the data flow is inconsistent between model training and inference. Figure 2 As shown in the diagram, the bypass pooling layer includes spatial pooling, content pooling, and linear layers. During training of the edge inference model, data flows into each layer, and the weights of the bypass pooling layer are remembered during this process. When the edge inference model performs inference, the entire bypass pooling layer is converted to a constant based on the weights remembered during training, and the data no longer passes through this layer. This saves computational overhead during edge inference and accelerates the inference efficiency of the edge inference model.

[0161] The pre-trained edge inference model provided in this invention is lightweight, saves computing resources while having good generalization ability, and effectively protects the privacy of user data, enabling the personalized learning process to be carried out safely and efficiently.

[0162] Figure 6 An exemplary block diagram of the model fine-tuning device for privacy protection according to the present invention is shown in one embodiment.

[0163] like Figure 6 As shown, in one embodiment of the present invention, a model fine-tuning device for privacy protection is proposed, comprising:

[0164] Embedding module 40 embeds the adaptive module into the feature extraction network to obtain an adaptive model;

[0165] The pre-training module 42 uses the first data to pre-train the adaptive model and updates the parameters of the adaptive model outside the adaptive module during the pre-training process to obtain the pre-trained model. Then, the pre-trained model is sent to the user terminal so that the user terminal can use the second data to train the pre-trained model and update the parameters of the adaptive module during the training process until the pre-trained model converges to obtain the user terminal inference model.

[0166] In the embedding module, this invention selects a feature extraction network model from the prior art as the initial model, and performs pre-training and fine-tuning based on the existing model structure and parameters, which can save a lot of time and cost for building and debugging the model.

[0167] In some embodiments, the adaptive module can be embedded into a feature extraction network built on a ResNet architecture.

[0168] Of course, in other embodiments, those skilled in the art can also use other types of feature extraction network models known to them, such as LeNet or VGG, as needed. The adaptive module typically contains adaptive network blocks or layers, used to embed into the existing feature extraction network model for small-sample-based model fine-tuning. The resulting adaptive model can more accurately serve specific samples and achieve the target task. Since the adaptive module is relatively independent of the original initial model, during training, only the parameters of the adaptive module can be updated, or only the parameters of the initial model can be updated, or all of the above parameters can be updated simultaneously. This allows for selective updating of model parameters according to different training objectives, reducing computational overhead. The number of adaptive modules can be one or more, determined by those skilled in the art based on the required feature extraction performance and computational capabilities of the adaptive model. By introducing adaptive modules, updating all parameters in the entire adaptive model during each training iteration is effectively avoided, aligning with the fine-tuning concept, reducing computational pressure, and improving the efficiency of adaptive model training.

[0169] In some embodiments, the adaptive module is built on linear or convolutional layers and embedded in the feature extraction network.

[0170] In some specific embodiments, a CaSE module is constructed based on convolutional layers as an adaptive module; the CaSE module includes convolutional layers and bypass pooling layers.

[0171] The CaSE module is an adaptive network block consisting of two convolutional layers and a side-pooling layer, with corresponding activation layers following the initial convolutional layers. Its key feature is that the data flow during edge inference model training and inference is inconsistent. The side-pooling layer includes spatial pooling, content pooling, and linear layers. During edge inference model training, data flows into each layer, and the weights of the side-pooling layer are memorized during this process. During edge inference model inference, the entire side-pooling layer is converted to a constant based on the weights memorized during training, and data no longer passes through this layer. This saves computational overhead during edge inference model inference and accelerates inference efficiency.

[0172] Of course, in other embodiments, adaptive modules can also be constructed using a linear layer-based approach.

[0173] In some embodiments, the adaptive module is embedded after the intermediate layer of the feature extraction network.

[0174] In some specific embodiments, the adaptive module can be embedded after the activation layer of two adjacent convolutional layers in a feature extraction network built on the ResNet architecture.

[0175] In the pre-training module, in some embodiments, the first data includes at least large data obtained from the server.

[0176] The big data obtained from the server includes various general public datasets. These datasets are characterized by their large quantity and wide coverage, which can ensure the training efficiency of the adaptive model and the feature representation effect of the obtained pre-trained model.

[0177] Using large amounts of data from the server to pre-train the adaptive model is to enhance the adaptability between the adaptive module and the initial model. This allows the adaptive model to retain the feature representation capabilities of the original feature extraction network while adapting to the changes in model structure brought about by embedding the adaptive module. The pre-trained model obtained at this time may not be consistent with the target task. It is only necessary to ensure that the pre-trained model has good feature representation capabilities. Therefore, this pre-training stage only updates the parameters of the adaptive model other than the adaptive module. This avoids updating all parameters of the entire model, which is in line with the concept of lightweight design and reduces the network and computational pressure on the server.

[0178] After the pre-training module distributes the pre-trained model to the user, the parameters of the adaptive module begin to update, while the other parameters of the pre-trained model are frozen as constants. At this time, the training process of the pre-trained model aims to improve the ability of personalized learning, which also avoids updating all the parameters of the entire model and improves training efficiency.

[0179] In some embodiments, the second data is personalized data obtained from the user's device.

[0180] In some embodiments, a second set of data is used to perform differential privacy training on the pre-trained model.

[0181] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the end-inference model is stolen, the user data it contains will not be leaked. It also resists model inversion attacks and membership inference attacks.

[0182] In one embodiment of the present invention, a model fine-tuning device for privacy protection is proposed, comprising:

[0183] The training module receives the pre-trained model from the server and trains the pre-trained model using the second data. During the training process, the parameters of the adaptive module are updated until the pre-trained model converges, thus obtaining the user-side inference model.

[0184] The pre-trained model is obtained by embedding the adaptive module into the feature extraction network on the server side, then using the first data to pre-train the adaptive model, and updating the adaptive model parameters other than the adaptive module during the pre-training process.

[0185] In the training module, in some embodiments, the second data is personalized data obtained from the user's terminal.

[0186] The user end includes end devices such as smartphones and smartwatches. These personal devices typically store the device owner's personalized data, such as consumption preferences, geographical location, and family relationships. The pre-trained model, trained using the first set of server-side data, possesses excellent feature representation capabilities. However, compared to the first set of data, the second set of user-specific data covers a much smaller scope, but contains far more precise and richer information. Fine-tuning using this user-specific data helps the pre-trained model quickly learn the preferences of specific users, ensuring the quality of personalized learning.

[0187] In the aforementioned device, the adaptive model has been pre-trained using big data on the server side, resulting in a pre-trained model. During the pre-training process, only the parameters outside the adaptive module were updated, with the aim of training the feature representation capability of the pre-trained model.

[0188] In some embodiments, the pre-trained model is trained using second data, and only the parameters of the adaptive module are updated during the training process, while the other parameters of the pre-trained model are treated as constants.

[0189] The parameters of the adaptive module are only updated on the user side, while the other parameters of the pre-trained model are frozen as constants after pre-training. The process of using second data to train the pre-trained model aims to improve the personalized learning ability of the edge inference model. This not only avoids updating all parameters of the entire model and improves training efficiency, but also enables targeted training of specific samples, thereby enhancing the accuracy of the edge inference model.

[0190] Preferably, in some embodiments, the pre-trained model is trained using second data for differential privacy.

[0191] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the edge inference model is stolen, the user's personalized data contained within it will not be leaked. It also resists model inversion attacks and membership inference attacks.

[0192] In some specific embodiments, differential privacy training of the pre-trained model is performed using second data, and the step of updating the parameters of the adaptive module during training includes:

[0193] Solve the sample-by-sample gradient based on the second data;

[0194] The gradient is processed sample by sample to obtain the processed gradient;

[0195] The pre-trained model is updated with the processed gradient.

[0196] The second dataset contains both user-specific data and corresponding data labels. After inputting the user-specific data into the pre-trained model, the loss is calculated based on the forward output and the corresponding data labels. The sample-by-sample gradient can then be obtained from the loss.

[0197] To protect users' personalized data, the differential privacy method randomizes the gradient for each sample. Even if the edge inference model is stolen, attackers cannot obtain the corresponding user personalized data based on the randomly generated gradient, thereby greatly reducing the risk of user information leakage and providing a relatively secure payment environment.

[0198] In some more specific embodiments, the step of processing the sample-by-sample gradient includes:

[0199] Clipping of gradients per sample;

[0200] The sample-by-sample gradients after cropping are summed, and noise is added to obtain the processed gradient.

[0201] It should be noted that a clipping norm C is used when clipping the sample-by-sample gradients. This means that every gradient with a L2 norm greater than C is scaled down to the size of C. After clipping all sample-by-sample gradients in the current batch, they are summed and noise is added to obtain the randomized gradient. This gradient is used to update the parameters of the pre-trained model, and then the next batch of training is performed until the model converges.

[0202] Optionally, in a more specific embodiment, the pre-trained model is trained using second data with differential privacy, and the steps for updating the parameters of the adaptive module during training are as follows:

[0203] Let ξ represent the sample of user personality data, and let b be the number of samples. Then the sample data can be represented as (ξ1, ξ2, ..., ξ). b The corresponding data label is represented by y, i.e., (y1, y2, ..., y3). b After inputting user personality data into the pre-trained model x, the model is then processed based on the forward output results (f(x, ξ1), f(x, ξ2), ..., f(x, ξ)). bThe loss is calculated using the data labels and the corresponding data labels. Based on the loss, the sample-by-sample gradient g can be obtained, i.e., (g) t，1 g t，2 , ..., g t，b ), where t represents the training batch.

[0204] Next, the gradient per sample is clipped according to the clipping norm, as follows:

[0205]

[0206] Among them, g t，i Let be the gradient of the i-th sample in the t-th batch; is the gradient after clipping; Clip() is the clipping function; C is the clipping norm.

[0207] After clipping all sample-by-sample gradients in the current batch, sum them and add noise to obtain the randomized gradient, as follows:

[0208]

[0209] in, The gradient is the result of the processing; N() is the noise function, σ is the attribute parameter in the noise function; b is the number of samples; i is the index of the sample.

[0210] The steps for updating the parameters of a pre-trained model using the processed gradients are as follows:

[0211]

[0212] Where x represents the pre-trained model; η represents the learning rate; x t This represents the model obtained from training the t-th batch.

[0213] After updating all parameters of the adaptive module in the pre-trained model according to the above formula, the next batch of training is carried out until the model converges.

[0214] The privacy-preserving model fine-tuning device described in this invention, by embedding an adaptive module, updates parameters outside the adaptive module on the server side and updates the parameters of the adaptive module during user-side training, thereby completing the fine-tuning of the end-to-end inference model. This saves significant computational resources, ensures the generalization ability of the end-to-end inference model, solves the overfitting problem, and effectively protects user privacy while providing personalized learning. Furthermore, it allows users to obtain a customized experience and improves the accuracy of the end-to-end inference model. In addition, the device employs differential privacy technology to effectively protect user data used for personalized learning; even if the end-to-end inference model is stolen, the user data it contains will not be leaked. It also resists model inversion attacks and member inference attacks.

[0215] In another embodiment of the present invention, a risk control device for protecting user privacy is provided, comprising:

[0216] The inference module inputs the acquired user-end personalized data into a pre-trained end-to-end inference model for inference to obtain personalized user information. The end-to-end inference model is trained using any of the model fine-tuning methods described above for privacy protection.

[0217] In the inference module, the user end includes end devices, such as smartphones, smartwatches, and other electronic devices. User-specific data is stored on these personal end devices, typically including the device owner's personal data, such as consumption preferences, geographical location, and family relationships. When the application a user is using connects to the internet and is granted certain data access permissions, the user's habits and preferences while using the application are remembered as personal data and stored on the end device. For example, when a user orders takeout using an application, the application can obtain the user's geographical location and dietary preferences; it can also obtain information about the user's lifestyle from their online shopping activity.

[0218] In fields such as mobile payments, the key to risk control lies in ensuring that while analyzing user needs and providing corresponding services after acquiring user's personalized data, user information leakage is avoided, thereby mitigating risks such as financial losses. After inputting user's personalized data, the edge inference model analyzes the data and infers a feature representation of the user's personalized data. This feature representation represents the user's personalized needs and preferences, allowing for the analysis of user needs based on the obtained user personalized information, providing personalized services, and enriching the user experience. Simultaneously, the edge inference model incorporates encryption technology during training, protecting user data and corresponding feature representations from leakage. This achieves a balance between enriching the user's personalized experience and protecting user data security, thus realizing risk control.

[0219] Preferably, in some embodiments, differential privacy training is employed when training the peer inference model.

[0220] Differential privacy is a cryptographic technique to prevent differential attacks. It aims to maximize the accuracy of query results while minimizing the possibility of an attacker identifying a single record when querying a statistical database by adding random perturbations to the query results. Differential privacy effectively protects user data used for personalized learning; even if the model is stolen, the user's personalized data contained within it will not be leaked. It also resists model inversion attacks and membership inference attacks.

[0221] To protect users' personalized data, the differential privacy method randomizes the gradient for each sample. Even if the edge inference model is stolen, attackers cannot obtain the corresponding user personalized data based on the randomly generated gradient, thereby greatly reducing the risk of user information leakage and providing a relatively secure payment environment.

[0222] In some embodiments, the user-end personalized data is input into a pre-trained end-to-end inference model that uses a CaSE module as an adaptive module. The CaSE module includes convolutional layers and bypass pooling layers. During inference, constants are used instead of bypass pooling layers.

[0223] It should be noted that the CaSE module is an adaptive network block consisting of two convolutional layers and a side-pooling layer. Its characteristic is that the data flow during edge inference model training and inference is inconsistent. Figure 2 As shown in the diagram, the bypass pooling layer includes spatial pooling, content pooling, and linear layers. During training of the edge inference model, data flows into each layer, and the weights of the bypass pooling layer are remembered during this process. When the edge inference model performs inference, the entire bypass pooling layer is converted to a constant based on the weights remembered during training, and the data no longer passes through this layer. This saves computational overhead during edge inference and accelerates the inference efficiency of the edge inference model.

[0224] The pre-trained edge inference model provided in this invention is lightweight, saves computing resources while having good generalization ability, and effectively protects the privacy of user data, enabling the personalized learning process to be carried out safely and efficiently.

[0225] One embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the steps of the model fine-tuning and risk control method for privacy protection described above in the present invention.

[0226] One embodiment of the present invention also provides another computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the steps of the model fine-tuning and risk control method for privacy protection described above in the present invention.

[0227] It should be noted that the above examples are merely specific embodiments of the present invention, and the present invention is obviously not limited to the above embodiments, with many similar variations. All modifications that can be directly derived or conceived by those skilled in the art from the content disclosed in this invention should fall within the protection scope of this invention.

Claims

1. A model fine-tuning method for privacy protection, applied on the server side, the method comprising: An adaptive module is constructed based on linear or convolutional layers, and the adaptive module is embedded into a feature extraction network to obtain an adaptive model. The adaptive model is pre-trained using the first data, and the parameters of the adaptive model other than the adaptive module are updated during the pre-training process to obtain the pre-trained model. The pre-trained model is distributed to the user terminal so that the user terminal can train the pre-trained model using the second data and update the parameters of the adaptive module during the training process until the pre-trained model converges to obtain the user terminal inference model.

2. The model fine-tuning method for privacy protection as described in claim 1, wherein the step of constructing the adaptive module based on linear layers or convolutional layers includes: A CaSE module is constructed based on convolutional layers to serve as the adaptive module; The CaSE module includes convolutional layers and bypass pooling layers.

3. The model fine-tuning method for privacy protection as described in claim 1, wherein the step of embedding the adaptive module into the feature extraction network includes: After embedding the adaptive module into the intermediate layer of the feature extraction network.

4. The model fine-tuning method for privacy protection as described in claim 1, wherein the step of embedding the adaptive module into the feature extraction network includes: The adaptive module is embedded in a feature extraction network built on the ResNet structure.

5. The model fine-tuning method for privacy protection as described in claim 4, wherein the step of embedding the adaptive module into a feature extraction network constructed based on the ResNet structure includes: The adaptive module is embedded after the activation layer of two adjacent convolutional layers in a feature extraction network built on the ResNet structure.

6. The model fine-tuning method for privacy protection as described in claim 1, wherein the first data includes at least big data obtained from the server.

7. A model fine-tuning method for privacy protection, applied to a user terminal, the method comprising: The system receives a pre-trained model from the server. The pre-trained model is obtained by the server building an adaptive module based on linear or convolutional layers, embedding the adaptive module into a feature extraction network, and then pre-training the adaptive model using the first data. During the pre-training process, the parameters of the adaptive model other than the adaptive module are updated. The pre-trained model is trained using the second data, and the parameters of the adaptive module are updated during the training process until the pre-trained model converges, thus obtaining the user-side inference model.

8. The model fine-tuning method for privacy protection as described in claim 7, wherein the step of training the pre-trained model using second data and updating the parameters of the adaptive module during training includes: The pre-trained model is trained using the second data. During the training process, only the parameters of the adaptive module are updated, while the other parameters of the pre-trained model are treated as constants.

9. The model fine-tuning method for privacy protection as described in claim 7, wherein the step of training the pre-trained model using the second data includes: The pre-trained model is trained using the second data for differential privacy.

10. The model fine-tuning method for privacy protection as described in claim 9, wherein the step of performing differential privacy training on the pre-trained model using second data and updating the parameters of the adaptive module during training includes: Solve the sample-by-sample gradient based on the second data; The sample-by-sample gradient is processed to obtain the processed gradient; The pre-trained model is updated with the processed gradient.

11. The model fine-tuning method for privacy protection as described in claim 10, wherein the step of processing the sample-by-sample gradient includes: The sample-by-sample gradient is clipped; The cropped gradients are summed sample by sample, and noise is added to obtain the processed gradient.

12. The model fine-tuning method for privacy protection as described in claim 7, wherein the second data is personalized data obtained from the user's terminal.

13. A risk control method for protecting user-end privacy, the method comprising: Obtain personalized data from the user's end; The user-end personalized data is input into a pre-trained terminal inference model for inference to obtain personalized user information, wherein the terminal inference model is trained using the method described in any one of claims 1 to 12.

14. The risk control method for protecting user privacy as described in claim 13, wherein the step of inputting the user-end personalized data into a pre-trained terminal inference model for inference includes: The user-end personalized data is input into a pre-trained end-to-end inference model that uses the CaSE module as an adaptive module. The CaSE module includes convolutional layers and bypass pooling layers. During inference, constants are used instead of bypass pooling layers.

15. A model fine-tuning device for privacy protection, comprising: An embedding module is constructed based on linear or convolutional layers to create an adaptive module, which is then embedded into a feature extraction network to obtain an adaptive model. The pre-training module uses the first data to pre-train the adaptive model and updates the parameters of the adaptive model other than the adaptive module during the pre-training process to obtain the pre-trained model. Then, the pre-trained model is distributed to the user terminal so that the user terminal can use the second data to train the pre-trained model and update the parameters of the adaptive module during the training process until the pre-trained model converges to obtain the user terminal inference model.

16. The model fine-tuning device for privacy protection as described in claim 15, wherein the embedding module constructs a CaSE module based on convolutional layers as the adaptive module, and embeds the adaptive module into the feature extraction network; the CaSE module includes convolutional layers and bypass pooling layers.

17. A model fine-tuning device for privacy protection, comprising: The training module receives a pre-trained model from the server and trains the pre-trained model using second data. During the training process, the parameters of the adaptive module are updated until the pre-trained model converges, thus obtaining the user-side inference model. The pre-trained model is obtained by constructing an adaptive module on the server side based on linear or convolutional layers, embedding the adaptive module into the feature extraction network, and then pre-training the adaptive model using the first data. During the pre-training process, the parameters of the adaptive model other than the adaptive module are updated.

18. The model fine-tuning device for privacy protection as described in claim 17, wherein during the training of the pre-trained model using the second data, the training module only updates the parameters of the adaptive module and treats the other parameters of the pre-trained model as constants.

19. The model fine-tuning device for privacy protection as described in claim 17, wherein the training module uses the second data to perform differential privacy training on the pre-trained model.

20. A risk control device for protecting user privacy, comprising: The inference module inputs the acquired user-end personalized data into a pre-trained end-to-end inference model for inference to obtain personalized user information, wherein the end-to-end inference model is trained using the method described in any one of claims 1 to 12.

21. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the methods described in claims 1-12.

22. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of any of the methods described in claims 1-12.

Images