Longitudinal federal learning privacy protection method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the field of federated learning, and discloses a longitudinal federated learning privacy protection method and device, equipment and a storage medium, and the method comprises the steps: obtaining a feature embedding vector corresponding to original data; calculating the sensitivity of the feature embedding vector relative to the original data based on the original data and the feature embedding vector, and calculating the gradient of the sensitivity with respect to the model parameters of the bottom model; uploading the feature embedding vector to a server, and receiving the gradient of the loss of the classification task, which is returned by the server, about the feature embedding vector; and combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and adopting a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model. The problem that the performance of the model cannot be ensured while the effect of defending the data reconstruction attack is ensured in the prior art is solved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of federated learning, and in particular, to a vertical federated learning privacy protection method, apparatus, device and storage medium. Background Art

[0002] The vertical federated learning framework can effectively ensure the joint training of multiple parties. However, there is currently a data reconstruction attack against vertical federated learning. The attacker has a trained vertical federated model and can reconstruct the original data based on the original intermediate results. This attack method will pose a major threat to vertical federated learning.

[0003] The existing technology for preventing data reconstruction attacks in the field of federated learning is differential privacy. Differential privacy protects privacy by adding noise. In federated learning, participants upload model parameters with added noise to the server to prevent sensitive data from being inferred. Differential privacy adds noise to the data, making the database query results insensitive to changes in individual records in the data set, thereby preventing attackers from inferring privacy attributes by using changes in statistical features. This method can effectively defend against data reconstruction attacks, but it may also affect the speed of model convergence and the accuracy of the model. Summary of the invention

[0004] The purpose of the present invention is to at least provide a vertical federated learning privacy protection method, device, equipment and storage medium, which can at least solve the problem that the existing solutions cannot ensure the performance of the model while ensuring the effect of defending against data reconstruction attacks, and at least can prevent the original data from being subjected to data reconstruction attacks while ensuring the normal progress of vertical federated learning training.

[0005] To solve the above technical problems, at least one embodiment of the present application provides a vertical federated learning privacy protection method, which is applied to a client, and updates the bottom model of the client in each round of communication between the client and the server until the bottom model converges. The method includes: initializing the bottom model of the client; obtaining original data, and inputting the obtained original data into the bottom model of the client to obtain a feature embedding vector corresponding to the original data; based on the original data and the feature embedding vector, calculating the sensitivity of the feature embedding vector relative to the original data, and calculating the gradient of the sensitivity with respect to the model parameters of the bottom model; uploading the feature embedding vector to the server, and receiving the gradient of the loss of the classification task returned by the server with respect to the feature embedding vector; combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and using the back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0006] In some optional embodiments, the sensitivity of the feature embedding vector relative to the original data is calculated based on the original data and the feature embedding vector, including: calculating the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix; calculating the L2 norm of the Jacobian matrix to obtain a scalar; and taking the product of the scalar and a pre-specified coefficient value as the sensitivity of the feature embedding vector relative to the original data.

[0007] In some optional embodiments, calculating the gradient of the sensitivity with respect to the model parameters of the bottom model includes: using the sensitivity to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0008] In some optional embodiments, the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector are combined, and the model parameters of the bottom model are updated by a back propagation method to obtain an updated bottom model, including: using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector to derive the model parameters of the bottom model to obtain the gradients corresponding to the model parameters of the bottom model; combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradients corresponding to the model parameters of the bottom model, and using a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0009] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned vertical federated learning privacy protection method.

[0010] At least one embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned vertical federated learning privacy protection method when executed by a processor.

[0011] At least one embodiment of the present application further provides a vertical federated learning privacy protection device, which is applied to a client, and updates the bottom model of the client in each round of communication between the client and the server until the bottom model converges, and the device includes: an initialization module, which is used to initialize the bottom model of the client;

[0012] A local training module, used for acquiring raw data, and inputting the acquired raw data into the bottom model of the client to obtain a feature embedding vector corresponding to the raw data, and uploading the feature embedding vector to the server;

[0013] A calculation module, configured to calculate, based on the original data and the feature embedding vector, a sensitivity of the feature embedding vector relative to the original data, and calculate a gradient of the sensitivity with respect to a model parameter of the bottom model;

[0014] An updating module is used to receive the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server, and to update the model parameters of the bottom model by using a back propagation method based on the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, so as to obtain an updated bottom model.

[0015] In some optional embodiments, the sensitivity of the feature embedding vector relative to the original data is calculated based on the original data and the feature embedding vector, including: calculating the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix; calculating the L2 norm of the Jacobian matrix to obtain a scalar; and taking the product of the scalar and a pre-specified coefficient value as the sensitivity of the feature embedding vector relative to the original data.

[0016] In some optional embodiments, calculating the gradient of the sensitivity with respect to the model parameters of the bottom model includes: using the sensitivity to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0017] In some optional embodiments, the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector are combined, and the model parameters of the bottom model are updated by a back propagation method to obtain an updated bottom model, including: using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector to derive the model parameters of the bottom model to obtain the gradients corresponding to the model parameters of the bottom model; combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradients corresponding to the model parameters of the bottom model, and using a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0018] A vertical federated learning privacy protection method, apparatus, device and storage medium of this embodiment initializes the bottom model of the client; then obtains the original data, and inputs the obtained original data into the bottom model of the client to obtain the feature embedding vector corresponding to the original data; and based on the original data and the feature embedding vector, calculates the sensitivity of the feature embedding vector relative to the original data, and calculates the gradient of the sensitivity with respect to the model parameters of the bottom model; uploads the feature embedding vector to the server, and receives the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server; combines the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and uses the back propagation method to update the model parameters of the bottom model to obtain an updated bottom model, thereby reducing the sensitivity of the feature embedding vector with respect to the original data to increase the difficulty of the client to reconstruct the original data using the original feature embedding vector data, thereby ensuring the privacy of the data, and compared with the current differential privacy scheme, the present invention does not add noise, has no impact on the accuracy of the original model, and ensures the accuracy of the original model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0020] Figure 1 is a flow chart of a vertical federated learning privacy protection method provided by an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of a vertical federated learning privacy protection device provided by another embodiment of the present application;

[0022] Figure 3 This is a schematic diagram of interaction between a server and N clients provided in another embodiment of the present application. DETAILED DESCRIPTION

[0023] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0024] In order to facilitate understanding of the embodiments of the present application, relevant content about federated learning is first introduced here.

[0025] Federated learning (FL) is an emerging distributed learning framework that facilitates data collaboration among multiple users without leaking their private data to each other. Currently, vertical federated learning (VFL), where the samples held by participants have the same sample space but different feature spaces, has received increasing attention. For example, a bank wants to build a machine learning model to evaluate whether a user's credit card application can be approved. If this goal is achieved through vertical federated learning, it requires the assistance of a fintech company. The bank has the features of "age" and "income", while the fintech company has the features of "deposit" and "average online shopping time". Only the bank has the label information in the training and test datasets, that is, the fact that indicates whether the user's credit card application should be approved. In order to train the vertical federated learning model, each participant iteratively exchanges certain intermediate results in a secure manner until a jointly trained model is obtained. Finally, the trained model will be released to each participant to prove the effectiveness and interpretability of the model. This vertical federated learning framework can effectively guarantee the joint training of multiple parties. However, there is currently a data reconstruction attack against vertical federated learning, where the attacker has a trained vertical federated model. Attackers can reconstruct the original data based on the original intermediate results. This type of attack poses a major threat to vertical federated learning.

[0026] The existing technology for preventing data reconstruction attacks in the field of federated learning is differential privacy. Differential privacy protects privacy by adding noise. In federated learning, participants upload model parameters with added noise to the server to prevent sensitive data from being inferred. Differential privacy adds noise to the data, making the database query results insensitive to changes in individual records in the data set, thereby preventing attackers from inferring privacy attributes by using changes in statistical features. This method can effectively defend against data reconstruction attacks, but it may also affect the speed of model convergence and the accuracy of the model.

[0027] The steps of the privacy protection scheme for vertical federated learning based on differential privacy are as follows:

[0028] The first step is model initialization: the client and server initialize their own bottom model and top model respectively.

[0029] The second step is local data conversion: each client uses its own bottom model locally to convert the original data into a feature embedding vector. The client adds noise to the feature embedding vector and then uploads the feature embedding vector to the server.

[0030] Step 3: Server model training: After receiving the feature embedding vectors of all participants, the server performs joint training and calculates the gradient of the loss function on the model parameters and feature embedding vectors of the top model. The server uses the gradient of the model parameters of the top model to back-propagate and update the top model, and returns the gradient of the feature embedding vector to each client.

[0031] Step 4: local model update: The client multiplies the gradient of the received feature embedding vector by the derivative of the feature embedding vector with respect to the bottom model to obtain the gradient of the model parameters of the bottom model, and backpropagates the bottom model based on the gradient of the model parameters of the bottom model to update the bottom model.

[0032] Repeat the above steps several times until the model converges.

[0033] The above method has the following disadvantages:

[0034] (1) Decreased accuracy: Since differential privacy requires the addition of noise, this may lead to a decrease in the accuracy of the model. This is because noise may mask some useful information, making the model inaccurate during both training and inference.

[0035] (2) Challenges of noise adjustment: Noise adjustment in differential privacy is a complex process that requires careful balance. If there is too much noise, the model performance may deteriorate; if there is too little noise, there may be a risk of privacy leakage.

[0036] In order to solve the above-mentioned technical problem of being unable to ensure the performance of the model while ensuring the effect of defending against data reconstruction attacks, the present invention proposes a vertical federated learning privacy protection method. The implementation details of the vertical federated learning privacy protection method of this embodiment are described in detail below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0037] Embodiment 1:

[0038] The vertical federated learning privacy protection method of this embodiment can be applied to electronic devices with communication, computing and data storage capabilities, and is specifically applied to the client. In each round of communication between the client and the server, the bottom model of the client is updated until the bottom model converges. The specific process of the method can be as follows: Figure 1 As shown, including:

[0039] Step S1, initializing the bottom model of the client.

[0040] Step S2, obtaining original data, and inputting the obtained original data into the bottom model of the client to obtain a feature embedding vector corresponding to the original data.

[0041] Step S3, based on the original data and the feature embedding vector, calculate the sensitivity of the feature embedding vector relative to the original data, and calculate the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0042] In some examples, step S3 “calculating the sensitivity of the feature embedding vector relative to the original data based on the original data and the feature embedding vector” includes:

[0043] Step S31, calculating the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix.

[0044] Step S32, calculating the L2 norm of the Jacobian matrix to obtain a scalar.

[0045] Specifically, the L2 norm, also known as the L2 norm, refers to the sum of the squares of the elements of the vector and then taking the square root. The L2 norm of the Jacobian matrix represents the sensitivity of the feature embedding vector to the original data.

[0046] Step S33: taking the product of the scalar and the pre-specified coefficient value as the sensitivity of the feature embedding vector relative to the original data.

[0047] Specifically, the L2 norm of the Jacobian matrix, i.e., the scalar, represents the sensitivity of the feature embedding vector relative to the original data. Here, the scalar is multiplied by a pre-specified coefficient value to obtain a numerical value, which is recorded as the sensitivity of the feature embedding vector relative to the original data.

[0048] In some examples, step S3 of "calculating the gradient of the sensitivity with respect to the model parameters of the bottom model" includes: using the sensitivity to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0049] Specifically, during training, the gradient of the sensitivity with respect to the model parameters of the bottom model can ensure that the model parameters of the bottom model are continuously updated until convergence. When the training is completed, it can ensure that the sensitivity of the feature embedding vector relative to the original data is greatly reduced, thereby preventing the original data from being subjected to data reconstruction attacks.

[0050] Step S4, uploading the feature embedding vector to the server, and receiving the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server.

[0051] Specifically, the server concatenates the feature embedding vectors uploaded by the clients of all participants and inputs them into the top model of the server. The server calculates the loss of the classification task based on the result output by the top model, and further calculates the gradient of the loss of the classification task with respect to the model parameters of the top model and the gradient of the feature embedding vector. The gradient of the loss of the classification task with respect to the model parameters of the top model is used to update the top model, and the gradient of the loss of the classification task with respect to the feature embedding vector is returned to the corresponding client.

[0052] Step S5, combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and using the back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0053] In some embodiments, step S5 "combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and updating the model parameters of the bottom model using a back propagation method to obtain an updated bottom model" includes:

[0054] Step S51, using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector, to derive the model parameters of the bottom model to obtain the gradient corresponding to the model parameters of the bottom model;

[0055] Step S52, combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient corresponding to the model parameters of the bottom model, and using a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0056] The method of this embodiment initializes the bottom model of the client; then obtains the original data, and inputs the obtained original data into the bottom model of the client to obtain the feature embedding vector corresponding to the original data; and based on the original data and the feature embedding vector, calculates the sensitivity of the feature embedding vector relative to the original data, and calculates the gradient of the sensitivity with respect to the model parameters of the bottom model; uploads the feature embedding vector to the server, and receives the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server; combines the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and uses the back propagation method to update the model parameters of the bottom model to obtain an updated bottom model, thereby reducing the sensitivity of the feature embedding vector with respect to the original data to increase the difficulty for the client to reconstruct the original data using the original feature embedding vector data, thereby ensuring the privacy of the data.

[0057] Compared with the differential privacy solution in the prior art, the method of this embodiment has the following advantages:

[0058] (1) High model accuracy. Compared with the differential privacy solution, the accuracy of the model in this embodiment is greatly improved. Since the differential privacy solution adds noise to the data, the noise will cover up some useful information, causing the accuracy of the model to decrease. However, this solution does not add noise and will not affect the accuracy of the original model.

[0059] (2) Effectively defend against data reconstruction attacks. When the attacker has a local model and the original feature embedding vector, the solution of this embodiment can reduce the sensitivity of the feature embedding vector relative to the original data, thereby defending against data reconstruction attacks.

[0060] Embodiment 2:

[0061] Another embodiment of the present application relates to a vertical federated learning privacy protection device. The implementation details of the vertical federated learning privacy protection device of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for the implementation of this solution. The schematic diagram of the vertical federated learning privacy protection device of this embodiment can be as follows Figure 2 As shown, it includes an initialization module 21, a local training module 22, a calculation module 23 and an update module 24.

[0062] The initialization module 21 is used to initialize the bottom model of the client.

[0063] The local training module 22 is used to obtain original data, and input the obtained original data into the bottom model of the client to obtain a feature embedding vector corresponding to the original data, and upload the feature embedding vector to the server.

[0064] The calculation module 23 is used to calculate the gradient of the sensitivity of the feature embedding vector relative to the original data based on the original data and the feature embedding vector, and calculate the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0065] In some examples, calculating the sensitivity of the feature embedding vector relative to the original data based on the original data and the feature embedding vector includes:

[0066] Step S231, calculating the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix.

[0067] Step S232, calculating the L2 norm of the Jacobian matrix to obtain a scalar.

[0068] Specifically, the L2 norm, also called the L2 norm, refers to the sum of the squares of the elements of the vector and then taking the square root. The L2 norm of the Jacobian matrix represents the sensitivity of the feature embedding vector to the original data.

[0069] Step S233: taking the product of the scalar and the pre-specified coefficient value as the sensitivity of the feature embedding vector relative to the original data.

[0070] Specifically, the L2 norm of the Jacobian matrix, i.e., the scalar, represents the sensitivity of the feature embedding vector relative to the original data. Here, the scalar is multiplied by a pre-specified coefficient value to obtain a numerical value, which is recorded as the sensitivity of the feature embedding vector relative to the original data.

[0071] In some examples, calculating the gradient of the sensitivity with respect to the model parameters of the bottom model includes: using the sensitivity to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

[0072] Specifically, during training, the gradient of the sensitivity with respect to the model parameters of the bottom model can ensure that the model parameters of the bottom model are continuously updated until convergence. When the training is completed, it can ensure that the sensitivity of the feature embedding vector relative to the original data is greatly reduced, thereby preventing the original data from being subjected to data reconstruction attacks.

[0073] The update module 24 is used to receive the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server, and to update the model parameters of the bottom model using a back propagation method based on the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, so as to obtain an updated bottom model.

[0074] In some embodiments, combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and updating the model parameters of the bottom model using a back propagation method to obtain an updated bottom model, comprises:

[0075] Step S241, using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector, to derive the model parameters of the bottom model to obtain the gradient corresponding to the model parameters of the bottom model;

[0076] Step S242, combining the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient corresponding to the model parameters of the bottom model, and using a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0077] Specifically, the server concatenates the feature embedding vectors uploaded by the clients of all participants and inputs them into the top model of the server. The server calculates the loss of the classification task based on the result output by the top model, and further calculates the gradient of the loss of the classification task with respect to the model parameters of the top model and the gradient of the feature embedding vector. The gradient of the loss of the classification task with respect to the model parameters of the top model is used to update the top model, and the gradient of the loss of the classification task with respect to the feature embedding vector is returned to the corresponding client. The client uses the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector to derive the model parameters of the bottom model to obtain the gradient corresponding to the model parameters of the bottom model, and then combines the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient corresponding to the model parameters of the bottom model, and uses the back propagation method to update the model parameters of the bottom model to obtain an updated bottom model.

[0078] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0079] Embodiment three:

[0080] In a specific application embodiment, N clients U = {u 1 ,u 2 ,…,u N}, take a server as an example, Figure 3 This is a schematic diagram of the interaction between the server and N clients, such as Figure 3 As shown, the vertical federated learning privacy protection method includes:

[0081] Step 1: Model initialization: server and client u i For the top model and bottom model B that you own respectively i Initialization is performed, where i=1,...,N.

[0082] Step 2: Local model training: Client u i The original data x i Input bottom model B i , get the feature embedding vector E i .

[0083] Step 3: Calculate sensitivity: Client u i Calculate the feature embedding vector data E i For the original data x i The first-order partial derivative of the Jacobian matrix G is obtained i , and calculate this Jacobian matrix G i L2 paradigm |G i | 2 , get a scalar, and multiply this scalar by a specified coefficient value reg, and record the resulting value as the feature embedding vector E i Relative to the original data x i sensitivity.

[0084] Step 4: Joint training: Client u i Upload feature embedding vector E i to the server. The server will receive all the client u i Uploaded feature embedding vector E i Splice and concatenate the feature vector E iAnd input to the top model of the server.

[0085] Step 5: Top model update: The server calculates the loss of the classification task based on the output of the top model, and further calculates the loss of the classification task with respect to the gradient of the model parameters of the top model and the feature embedding vector E i The gradient of the loss of the classification task with respect to the model parameters of the top model is used to update the top model, and the loss of the classification task with respect to the feature embedding vector E i The gradient (gradient 2) is returned to the corresponding client.

[0086] Step 6: Update the bottom model: Use the sensitivity to derive the model parameters of the bottom model to obtain the gradient (gradient 1) of the sensitivity with respect to the model parameters of the bottom model. Use the loss of the classification task with respect to the feature embedding vector E i The gradient (gradient 2) and the feature embedding vector E i The product of is used to derive the model parameters of the bottom model to obtain the gradients corresponding to the model parameters of the bottom model. The model parameters of the bottom model are updated by using the back propagation method, combining the gradient of the sensitivity with respect to the model parameters of the bottom model (gradient 1) and the gradient corresponding to the model parameters of the bottom model to obtain an updated bottom model.

[0087] Repeat the above steps for multiple rounds until the top model and bottom model converge.

[0088] Embodiment 4:

[0089] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vertical federated learning privacy protection method in the above-mentioned embodiments.

[0090] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0091] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0092] Embodiment five:

[0093] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0094] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0095] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A vertical federated learning privacy protection method, characterized in that: Applied to a client, in each round of communication between the client and the server, the bottom model of the client is updated until the bottom model converges, the method comprising: Initializing the bottom model of the client; Acquire original data, and input the acquired original data into the bottom model of the client to obtain a feature embedding vector corresponding to the original data; Based on the original data and the feature embedding vector, calculating the sensitivity of the feature embedding vector relative to the original data, and calculating the gradient of the sensitivity with respect to the model parameters of the bottom model; Uploading the feature embedding vector to the server, and receiving the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server; The gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector are combined, and the model parameters of the bottom model are updated using a back-propagation method to obtain an updated bottom model.

2. The vertical federated learning privacy protection method according to claim 1 is characterized in that: The calculating, based on the original data and the feature embedding vector, the sensitivity of the feature embedding vector relative to the original data comprises: Calculate the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix; Calculate the L2 norm of the Jacobian matrix to obtain a scalar; The product of the scalar and the pre-specified coefficient value is taken as the sensitivity of the feature embedding vector relative to the original data.

3. The privacy protection method for vertical federated learning according to claim 1 is characterized in that: The calculating the gradient of the sensitivity with respect to the model parameters of the bottom model comprises: The sensitivity is used to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

4. The privacy protection method for vertical federated learning according to claim 1 is characterized in that: The method combines the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and uses a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model, including: Using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector, the model parameters of the bottom model are derived to obtain the gradient corresponding to the model parameters of the bottom model; The gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient corresponding to the model parameters of the bottom model are combined, and the model parameters of the bottom model are updated using a back propagation method to obtain an updated bottom model.

5. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the vertical federated learning privacy protection method as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vertical federated learning privacy protection method described in any one of claims 1 to 4 is implemented.

7. A vertical federated learning privacy protection device, characterized in that: Applied to a client, the bottom model of the client is updated in each round of communication between the client and the server until the bottom model converges, and the device includes: An initialization module, used for initializing the bottom model of the client; A local training module, used for acquiring raw data, and inputting the acquired raw data into the bottom model of the client to obtain a feature embedding vector corresponding to the raw data, and uploading the feature embedding vector to the server; A calculation module, configured to calculate, based on the original data and the feature embedding vector, a sensitivity of the feature embedding vector relative to the original data, and calculate a gradient of the sensitivity with respect to a model parameter of the bottom model; An updating module is used to receive the gradient of the loss of the classification task with respect to the feature embedding vector returned by the server, and to update the model parameters of the bottom model by using a back propagation method based on the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, so as to obtain an updated bottom model.

8. The vertical federated learning privacy protection device according to claim 7, characterized in that: The calculating, based on the original data and the feature embedding vector, the sensitivity of the feature embedding vector relative to the original data comprises: Calculate the first-order partial derivative of the feature embedding vector with respect to the original data to obtain a Jacobian matrix; Calculate the L2 norm of the Jacobian matrix to obtain a scalar; The scalar and the pre-specified coefficient value are used as the sensitivity of the feature embedding vector relative to the original data.

9. The vertical federated learning privacy protection device according to claim 7, characterized in that: The calculating the gradient of the sensitivity with respect to the model parameters of the bottom model comprises: The sensitivity is used to derive the model parameters of the bottom model to obtain the gradient of the sensitivity with respect to the model parameters of the bottom model.

10. The vertical federated learning privacy protection device according to claim 7, characterized in that: The method combines the gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient of the loss of the classification task with respect to the feature embedding vector, and uses a back propagation method to update the model parameters of the bottom model to obtain an updated bottom model, including: Using the product of the gradient of the loss of the classification task with respect to the feature embedding vector and the feature embedding vector, the model parameters of the bottom model are derived to obtain the gradient corresponding to the model parameters of the bottom model; The gradient of the sensitivity with respect to the model parameters of the bottom model and the gradient corresponding to the model parameters of the bottom model are combined, and the model parameters of the bottom model are updated using a back propagation method to obtain an updated bottom model.

Images