A data label protection method, device and medium in vertical federated learning

Abstract

The present invention discloses a data label protection method, device, and medium for vertical federated learning, belonging to the field of data security. For use in a vertical federated learning system, before the first round of training, the ciphertext layer list, the list of layers to be reconstructed, and selection criteria parameters are initialized. The method includes: Step 1: Model layer masking based on secret sharing, including secret sharing and reconstruction processing; Step 2: Model security update: including secure forward propagation and backpropagation; Step 3: Shadow model update: Locally constructing a training shadow model and calculating the gradient value; Step 4: Selecting and adjusting the model layer to be masked according to a preset privacy budget threshold, and updating the ciphertext layer list and the list of layers to be reconstructed according to the adjusted masked model layer; Step 5: Re-executing the model layer masking process of Step 1 using the new ciphertext layer list and the list of layers to be reconstructed, starting the next round of training, and continuing until completion. This method can improve the data label security in vertical federated learning.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a data label protection method in vertical federated learning. Background Art

[0002] In vertical federated learning systems, participants' private data labels are at risk of being stolen. Attackers can exploit intermediate data passed during model training (such as feature embeddings, feature embedding gradients, and model gradients) to infer the data labels of other participants. This attack poses a significant threat to the data security of all participants and severely hinders the promotion and implementation of vertical federated learning technology in practical applications.

[0003] To address the problem of data label leakage in vertical federated learning, various technologies have been proposed to address this issue. These technologies can be broadly categorized into the following four types:

[0004] a. Perturbation-based methods: These methods primarily reduce the correlation between intermediate data and private data labels by adding perturbations to the intermediate data. For example, perturbations are added to feature embeddings or noise is injected into the gradients of feature embeddings.

[0005] b. Obfuscation-based methods: These methods directly perturb private data labels. For example, they utilize differential privacy techniques to perturb labels offline and then use the processed labels for model training.

[0006] c. Regularization-based methods: This type of method introduces a regularization term into the training loss function of federated learning to constrain the correlation between intermediate data and privacy labels, thereby reducing the risk of label leakage.

[0007] d. Cryptography-based methods: These methods typically employ cryptographic methods, such as using homomorphic encryption to encrypt intermediate data or utilizing secure multi-party computing to design privacy-preserving model training protocols.

[0008] While the aforementioned methods have mitigated the data label leakage issue to some extent, practical applications still face bottlenecks such as performance overhead, precision loss, and implementation complexity. Therefore, balancing efficiency and model performance while ensuring data security remains a key research topic in the field of vertical federated learning.

[0009] Existing technologies for privacy data label protection in vertical federated learning mainly face the following three problems: (1) It is difficult to balance model performance and data privacy. Existing technologies usually need to adjust hyperparameters to strike a balance between model performance and data privacy. However, in practical applications, it is often difficult to take both into account at the same time: in order to obtain higher model performance, it is usually necessary to sacrifice privacy protection effects; and in order to achieve stronger privacy protection, the model performance will be significantly reduced. Therefore, it is difficult to find an optimal balance point. (2) The computational cost is too high and difficult to implement in practice. Cryptography-based data label protection methods, such as homomorphic encryption or secure multi-party computation, are usually accompanied by a large amount of computational cost and communication overhead. These high costs are often unacceptable in practical application scenarios, hindering the promotion and implementation of the technology. (3) Lack of adjustable privacy protection options. In practical applications, users usually need to set different privacy protection thresholds based on their own privacy needs and computing resources. However, existing technologies cannot provide users with flexible privacy budget options and lack controllability over the degree of privacy protection.

[0010] In view of this, the present invention is proposed. Summary of the Invention

[0011] The purpose of the present invention is to provide a data label protection method, device and medium in vertical federated learning, which can improve model performance as much as possible while ensuring data privacy, ensure efficient computing and communication, and provide flexible privacy budget options, thereby solving the above-mentioned technical problems existing in the prior art.

[0012] The purpose of the present invention is achieved through the following technical solutions:

[0013] A data label protection method in vertical federated learning. Before starting model training, the privacy protector initializes the ciphertext layer list, the list of layers to be reconstructed, and the selection criterion parameters. Then, training begins, including:

[0014] Step 1: Perform secret sharing masking on the corresponding model layer parameters in the ciphertext layer list;

[0015] Perform secret reconstruction on the corresponding model layer parameters in the list of layers to be reconstructed;

[0016] Step 2: Model security update:

[0017] Secure forward propagation: If the model layer is a ciphertext layer, secret sharing and secret reconstruction are performed on the input to obtain the input for the next model layer. The above process is iterated until the final feature embedding of the output model is obtained.

[0018] Secure backpropagation: Calculate the global model loss and perform backpropagation from the global output layer of the global model. If the model layer is a ciphertext layer, perform secret sharing and secret reconstruction on the input gradient value to obtain the plaintext gradient value of the previous model layer. Iterate the above process until the parameters of each layer of the model are updated.

[0019] Step 3, shadow model update:

[0020] The privacy protector builds and trains the shadow model locally and calculates the gradient value of the shadow model;

[0021] Step 4: Select and adjust the model layer to be masked according to the preset privacy budget threshold, and update the new ciphertext layer list and the new to-be-reconstructed layer list;

[0022] Step 5: Return to step 1 with the new ciphertext layer list and the new layer list to be reconstructed to start the next round of training until the training is completed.

[0023] A processing device comprising:

[0024] at least one memory for storing one or more programs;

[0025] At least one processor is capable of executing one or more programs stored in the memory. When the one or more programs are executed by the processor, the processor is enabled to implement the method described in the present invention.

[0026] A readable storage medium stores a computer program, which can implement the method described in the present invention when the computer program is executed by a processor.

[0027] Compared with the prior art, the data label protection method, device, and medium in vertical federated learning provided by the present invention have the following beneficial effects:

[0028] First, the method of the present invention uses secret sharing technology to mask parameters of important model layers, minimizing the leakage of label privacy. Secret sharing ensures that the ciphertext and plaintext calculation results remain consistent during linear operations, thereby minimizing the impact on model accuracy while protecting privacy. This technical feature enables the method to achieve a good balance between privacy protection and model performance.

[0029] Secondly, by selectively adjusting the model layers to be masked according to a preset privacy budget threshold, the proposed method dynamically identifies and selects the model layers most critical to label privacy protection, applying secret sharing techniques only to these layers. This selective masking mechanism effectively avoids masking all model layers, significantly reducing computational and communication overhead. Furthermore, by precisely selecting critical layers, the method not only improves the efficiency of privacy protection but also enhances its practicality.

[0030] Furthermore, the method of the present invention allows users to flexibly set the label privacy budget according to actual needs, providing a high degree of adjustability. When the privacy budget is high, only a small number of model layers need to be selected for masking, which can both meet privacy protection requirements and maximize model performance. In contrast, when the privacy budget is low, more model layers can be selected for masking, providing stronger privacy protection capabilities. This flexibility enables the method to adapt to different application scenarios and user needs.

[0031] This invention effectively addresses the shortcomings of existing data labeling privacy protection methods, significantly improving privacy protection capabilities, model performance, and computational and communication efficiency. Leveraging secret sharing and a dynamic layer selection mechanism, this method effectively enhances data labeling privacy while ensuring model performance stability. By significantly reducing computational and communication overhead, its efficiency and practicality are further improved.

[0032] Finally, the proposed method is concise and easy to implement, making it applicable to a variety of vertical federated learning scenarios. Its efficient data privacy protection capabilities and flexible privacy budget settings make it promising for broad application in fields such as finance, healthcare, and smart cities that require data collaboration but place a high priority on privacy protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 Flowchart of a data label protection method in vertical federated learning provided by an embodiment of the present invention.

[0035] Figure 2 A specific flow chart of the data label protection method in vertical federated learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the specific content of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments, and do not constitute a limitation of the present invention. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] First, the following terms may be used in this article:

[0038] The term “and / or” means that either or both of them can be realized at the same time. For example, X and / or Y includes both “X” or “Y” and “X and Y”.

[0039] The terms "include," "comprises," "contains," "has," or other similar expressions should be interpreted as non-exclusive. For example, "including certain technical features (such as raw materials, components, ingredients, carriers, dosage forms, materials, dimensions, parts, components, mechanisms, devices, steps, procedures, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products, or manufactured articles)" should be interpreted as including not only the technical features explicitly listed, but also other technical features known in the art that are not explicitly listed.

[0040] The term "consisting of" excludes any technical features not explicitly listed. If used in a claim, this term renders the claim closed, excluding any technical features other than those explicitly listed, except for conventional impurities associated with them. If this term appears only in a clause of a claim, it limits only the elements explicitly listed in that clause; elements listed in other clauses are not excluded from the claim as a whole.

[0041] Unless otherwise specified or limited, the terms "mounted," "connected," "connect," and "fixed" should be interpreted broadly. For example, they can refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this document based on specific circumstances.

[0042] When concentration, temperature, pressure, size or other parameters are expressed in the form of a numerical range, the numerical range should be understood to specifically disclose all ranges formed by the pairing of any upper limit, lower limit, or preferred value within the numerical range, regardless of whether the range is explicitly stated. For example, if a numerical range of "2 to 8" is stated, the numerical range should be interpreted as including ranges of "2 to 7," "2 to 6," "5 to 7," "3 to 4 and 6 to 7," "3 to 5 and 7," "2 and 5 to 7," etc. Unless otherwise specified, the numerical ranges stated herein include both their endpoints and all integers and fractions within the numerical range.

[0043] The terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings and are only for the convenience and simplification of description, and do not explicitly or implicitly indicate that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as a limitation to this document.

[0044] The scheme provided by the present invention is described in detail below. The contents not described in detail in the examples of the present invention belong to the prior art known to professionals in this field. If specific conditions are not specified in the examples of the present invention, they are carried out according to conventional conditions in the field or conditions recommended by the manufacturer. If the manufacturer of the reagents or instruments used in the examples of the present invention is not specified, they are all conventional products that can be purchased commercially.

[0045] like Figure 1 、 Figure 2 As shown, an embodiment of the present invention provides a data label protection method in vertical federated learning, which is used in a vertical federated learning system in which multiple participants perform vertical federated learning and train a vertical federated model. Before the first round of training of the vertical federated model, the privacy protector initializes the ciphertext layer list, the to-be-reconstructed layer list, and the selection criterion parameters, and then starts training, including:

[0046] Step 1: Perform secret sharing masking on the corresponding model layer parameters in the ciphertext layer list;

[0047] Perform secret reconstruction on the corresponding model layer parameters in the list of layers to be reconstructed;

[0048] Step 2: Model security update:

[0049] Secure forward propagation: If the model layer is a ciphertext layer, secret sharing and secret reconstruction are performed on the input to obtain the input for the next model layer. The above process is iterated until the final feature embedding of the output model is obtained.

[0050] Secure backpropagation: Calculate the global model loss and perform backpropagation from the global output layer of the global model. If the model layer is a ciphertext layer, perform secret sharing and secret reconstruction on the input gradient value to obtain the plaintext gradient value of the previous model layer. Iterate the above process until the parameters of each layer of the model are updated.

[0051] Step 3, shadow model update:

[0052] The privacy protector builds and trains the shadow model locally and calculates the gradient value of the shadow model;

[0053] Step 4: Select and adjust the model layer to be masked according to the preset privacy budget threshold, and update the new ciphertext layer list and the new to-be-reconstructed layer list;

[0054] Step 5: Return to step 1 with the new ciphertext layer list and the new layer list to be reconstructed to start the next round of training until the training is completed.

[0055] In step 5 of the above method, the way to determine whether the model training is completed is: one is that the number of training rounds of the model has reached a preset number of rounds; the other is that the loss value calculated by the previous and next rounds of the model is less than a preset threshold.

[0056] Preferably, in step 1 of the above method, the corresponding model layer parameters in the ciphertext layer list are masked for secret sharing in the following manner:

[0057] Decompose the model layer parameters corresponding to the ciphertext layer list into shared parameter 1 and shared parameter 2 through secret sharing, and add noise to shared parameter 2 to obtain noisy shared parameter 2;

[0058] In step 1, the complete model layer parameters corresponding to the reconstructed layer list are not secretly reconstructed, and the incomplete model layer parameters corresponding to the reconstructed layer list are secretly reconstructed in the following manner, including:

[0059] By summing, a noise parameter is introduced into the shared parameter 2 to obtain the noisy shared parameter 2, and the shared parameter 1 and the noisy shared parameter 2 are summed to reconstruct the model layer parameters;

[0060] In the secure forward propagation of step 2, if the model layer is a plaintext layer, the forward propagation method of traditional vertical federated learning is used for calculation;

[0061] In the secure backpropagation of step 2, if the model layer is a plaintext layer, the backpropagation method of traditional vertical federated learning is used for calculation;

[0062] In step 4, the model layers to be masked are selected and adjusted according to the preset privacy budget threshold in the following manner, and a new ciphertext layer list and a new list of layers to be reconstructed are updated, including:

[0063] The privacy protector locally simulates the degree of data label leakage of the current model through a shadow model. Based on the degree of data label leakage and the privacy budget threshold preset by the privacy protector, the privacy protector adds a model layer for parameter masking so that the degree of data label leakage of the masked model is less than the privacy budget threshold or all parameters of the model layers are masked. The privacy protector updates the new ciphertext layer list and the new list of layers to be reconstructed according to the adjusted masked model layer.

[0064] Preferably, in the above method, the ciphertext layer list, the to-be-reconstructed layer list, and the selection criterion parameters are initialized in the following manner, including:

[0065] The initialized ciphertext layer list is , Indicates the model layer in the model trained by the privacy protection party that requires secret sharing for masking;

[0066] The initialized list of layers to be reconstructed is , V=[L] - {1} represents the model layer that needs to be secretly reconstructed in the privacy-preserving party’s trained model, where is the total number of model layers;

[0067] Initialization selection criterion parameters , in subsequent training, it is used to continuously accumulate the gradient value of the model to the selection criterion parameter middle.

[0068] Preferably, in step 1 of the above method, the ciphertext layer list is received by the model layer masking module set by the privacy protection party , for the ciphertext layer list The corresponding model layer parameters in the secret sharing masking process include:

[0069] The model layer masking module uses secret sharing to mask the ciphertext layer list. Elements The corresponding model Layer Parameters Decompose into shared parameters and shared parameter 2 , by summing Shared parameter 2 Introducing noise parameters , get the noise-added shared parameter 2;

[0070] In step 2, the model layer masking module receives the list of layers to be reconstructed , treat the reconstruction layer list The corresponding model layer parameters in are secretly reconstructed, including:

[0071] The model layer masking module determines the list of layers to be reconstructed Elements The corresponding model Layer Parameters Is it complete? If it is complete, no processing is performed. Otherwise, the layer parameters are secretly reconstructed as follows: by summing Shared parameter 2 Noise parameters are introduced , get the noise sharing parameter 2 , sum the shared parameter 1 and the noise-added shared parameter 2 , reconstruct the plaintext model Layer Parameters .

[0072] Preferably, in the above method, in the secure forward propagation of step 2, if the model layer is a ciphertext layer, the input is subjected to secret sharing and secret reconstruction processing in the following manner to obtain the input of the next model layer, including:

[0073] Decompose the input of the model layer into two shared input values ​​through secret sharing. Input each shared input value into the corresponding shared parameter of the model layer to obtain the corresponding two shared output values ​​of the model layer. Secretly reconstruct the two shared output values ​​in the following manner to obtain the input of the next model layer.

[0074] The way to secretly reconstruct the two shared output values ​​to get the input of the next model layer is:

[0075] Add noise to the shared output value 2 corresponding to the shared parameter 2 of the model layer to obtain the noisy shared output value 2, and sum the shared output value 1 corresponding to the shared parameter 1 of the model layer and the noisy shared output value 2 to obtain the input of the next model layer;

[0076] In the secure backpropagation of step 2, if the model layer is a ciphertext layer, the plaintext gradient value of the previous model layer is obtained by performing secret sharing and secret reconstruction on the input gradient value in the following manner, including:

[0077] The input gradient value of the model layer is decomposed into two shared gradient values ​​through secret sharing. The two shared gradient values ​​are respectively input into the corresponding shared parameters of the model layer to obtain the corresponding two shared gradient values ​​of the previous layer of the model layer. The two shared gradient values ​​of the previous layer are secretly reconstructed in the following way to obtain the plaintext gradient value of the previous model layer;

[0078] The method of secretly reconstructing the two shared gradient values ​​of the previous layer to obtain the plaintext gradient value of the previous model layer is:

[0079] Add noise to the previous layer shared gradient value two corresponding to the shared parameter two of the model layer to obtain the noisy previous layer shared gradient value two, and sum the previous layer shared gradient value one corresponding to the shared parameter one of the model layer and the noisy previous layer shared gradient value two to obtain the plaintext previous model layer gradient value.

[0080] Preferably, in the above method, in step 3, the privacy protection party constructs and trains the shadow model locally in the following manner to calculate the gradient value of the shadow model, including:

[0081] The privacy-preserving party constructs a shadow model locally with the same structure as the attacker's model but with different parameters. And use the auxiliary data set to train the shadow model locally in plain text. After training, perform back propagation calculation on the shadow model to obtain the shadow model The gradient value of .

[0082] Preferably, in step 4 of the above method, the privacy protection party selects and adjusts the model layer to be masked according to a preset privacy budget threshold through the model layer selection module in the following manner, including:

[0083] The privacy protection side selects the shadow model from the model layer The gradient value is accumulated to the selection criterion parameter In the selection criteria parameter The accumulated gradient values ​​of each model layer in are arranged in descending order;

[0084] Privacy protection is achieved by selecting modules at the model layer through shadow models and auxiliary data sets, simulate the data label leakage degree of the current model locally, if the data label leakage degree is judged to be higher than the privacy budget threshold set by the privacy protection party, then select the criterion parameters from the descending order The parameters of the model layer with the largest cumulative gradient value are selected for masking, and the above process is repeated iteratively until the degree of leakage of the masked model data label is less than the set privacy budget threshold or the selection criterion parameter There are no model layers to choose from.

[0085] Preferably, in step 4 of the above method, the privacy protector updates the new ciphertext layer list and the new to-be-reconstructed layer list in the following manner, including:

[0086] The privacy protection party will select the model layers that need to be masked to form a new ciphertext layer list , the model layers that do not need masking are not selected to form a new list of layers to be reconstructed , the new ciphertext layer list and the new list of layers to be refactored Passed to the model-level masking module for model-level masking processing for secret sharing.

[0087] An embodiment of the present invention further provides a processing device, comprising:

[0088] at least one memory for storing one or more programs;

[0089] At least one processor is capable of executing one or more programs stored in the memory. When the one or more programs are executed by the processor, the processor is enabled to implement the method described in the present invention.

[0090] An embodiment of the present invention further provides a readable storage medium storing a computer program, which can implement the method of the present invention when executed by a processor.

[0091] In summary, the method of the embodiment of the present invention can effectively overcome the limitations of the existing technology and effectively solve the three major problems faced by the existing technology: (1) The optimal trade-off between model performance and data privacy. The method of the present invention can improve the model performance as much as possible while ensuring data privacy, solving the problem that the existing technology is difficult to balance the two. (2) Efficient computing and communication. The method is designed with low computing and communication overheads, and can run efficiently in actual application scenarios, thus having good feasibility. (3) Flexible privacy budget options. The method of the present invention provides users with adjustable privacy budget options. Users can flexibly adjust the privacy budget value according to their own privacy needs and computing resources to achieve different levels of privacy protection.

[0092] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the solution provided by the embodiment of the present invention is described in detail with reference to specific embodiments below.

[0093] Example 1

[0094] This embodiment provides a data label protection method in vertical federated learning. The method achieves efficient data label privacy protection by performing secret sharing and secret reconstruction of model parameters, combined with dynamic model layer selection and noise injection mechanism.

[0095] like Figure 2 As shown, the data label protection method of this embodiment adds two modules and two calculation processes compared to the traditional vertical federated learning framework, which are described in detail as follows:

[0096] ① Secret Sharing-Based Model-Layer Masking Module: This module uses secret sharing to share and reconstruct the attacker's model-layer parameters based on the ciphertext layer list and the list of layers to be reconstructed. The ciphertext layer parameters are divided into multiple shared values ​​(Shares) through secret sharing, effectively preventing the attacker from recovering the original parameters. For ciphertext layer parameters that need to be reconstructed, the target parameters are reconstructed by summing the shared values.

[0097] ② Model Security Update Process: This process is used to train the vertical federated learning model. During training, the calculation process for the plaintext layer parameters is consistent with traditional vertical federated learning. For the ciphertext layer parameters, secret sharing technology is used to complete the forward and back propagation calculations to ensure the privacy of data labels during training.

[0098] ③ Shadow Model Update Process: This process is performed locally by the privacy protector. By constructing a shadow model, the privacy protector can dynamically monitor the potential privacy leakage of data labels. The shadow model's structure is identical to the attacker's model, but it is trained locally using an auxiliary dataset. By calculating the parameter gradients of the shadow model, the privacy protector can estimate the gradient distribution of the attacker's true model and thus assess the current level of data leakage.

[0099] ④ Model layer selection module with adjustable privacy budget: This module dynamically determines which model layers to mask based on the user-set label privacy budget threshold and the model parameter gradients calculated from the shadow model. After selection, it generates a new list of ciphertext layers and a list of layers to be reconstructed, and sends these lists to the model layer masking module for the next round of vertical federated model training.

[0100] like Figure 1 As shown, the method specifically includes the following steps:

[0101] Parameter initialization:

[0102] Before the first round of training of the vertical federation model, the ciphertext layer list is initialized. and the list of layers to be reconstructed ,in, Indicates the model layer that requires secret sharing in the model; V=[L] - {1} : Indicates the model layer that needs to be secretly reconstructed, where is the total number of model layers.

[0103] Subsequently, these two lists are passed to the model layer masking module so that the model layer masking module can be used to mask the ciphertext layer list. Secret sharing of model layer parameters; according to the list of layers to be reconstructed Perform secret reconstruction of model layer parameters.

[0104] Next, initialize the selection criteria parameters During the training of the vertical federated model, the model gradient value will continue to accumulate to the selection criterion parameter In the following model layer selection module, the selection criteria parameters The important model layers are dynamically selected for masking to enhance the privacy protection effect.

[0105] Step 1: Model-level masking based on secret sharing:

[0106] The model layer masking module is used to perform model layer masking based on secret sharing. The core function of this module is to encrypt and protect the parameters of different model layers through secret sharing and reconstruction mechanism, as shown below.

[0107] Secret Sharing: This process receives a list of ciphertext layers , for the list Each element in , the corresponding model layer Parameters are shared secretly. Secret sharing decomposes the model parameters into two shared parameters (Share): shared parameter one and shared parameter 2 To prevent attackers from using two shared parameters to reconstruct the original parameters, the two shared parameters are summed up. Introducing noise , get the noise sharing parameter 2 .

[0108] Secret reconstruction: This process receives a list of ciphertext layers , for the list Each element in , the corresponding model layer Similarly, to prevent attackers from recovering the original model parameters through the reconstructed parameters, the shared parameters are summed up. Noise introduced , get the noise sharing parameter 2 Finally, the sum of the two shared parameters can restore the model's Layer Parameters .

[0109] Through the above-mentioned secret sharing and secret reconstruction mechanism, the privacy of model layer parameters during transmission and calculation is ensured, and the protection effect is further enhanced by combining noise injection.

[0110] Step 2, model security update process:

[0111] The secure update process of the model is divided into two sub-processes: secure forward propagation and secure back propagation, where

[0112] Secure forward propagation: If the current model layer is a plaintext layer, the forward propagation method of traditional vertical federated learning is used for computation: the input of this layer is fed into the plaintext layer to obtain the output of this layer. If the current model layer is a ciphertext layer, the following steps are performed: the input of this layer is decomposed into two shared values ​​(Shares) using a secret sharing mechanism. Each shared value is input into the shared parameters of the corresponding model layer parameters to obtain the two output shared values ​​(Shares) of this layer. Finally, the two output shared values ​​are reconstructed to obtain the input of the next model layer. This process is iterated continuously until the feature embedding of the final output model is achieved.

[0113] Secure backpropagation: First, calculate the loss value of the global model and perform backpropagation calculations starting from the global output layer. For the plaintext layer: Calculate according to the traditional federated learning method, propagate the gradient value of this layer forward in plaintext, and obtain the gradient value of the previous plaintext layer. For the ciphertext layer: The following steps need to be performed: Decompose the input gradient value of this layer into two shared gradient values ​​(Share) through a secret sharing mechanism, and input these two shared gradient values ​​into the shared parameters of the corresponding model layer parameters respectively to obtain the shared gradient value (Share) of the previous layer corresponding to each model layer. Finally, secretly reconstruct the two shared gradient values ​​to obtain the gradient value of the previous plaintext layer. The above process is iterated continuously until the parameters of each layer of the model are updated.

[0114] Step 3, shadow model update process:

[0115] Shadow model update is a process performed locally by the privacy protector to dynamically adjust the parameter masking strategy of the model layer to enhance the privacy protection effect.

[0116] The specific steps are as follows: The privacy protection party builds a shadow model locally The shadow model has the same structure as the attacker's model, but with different parameters. The shadow model is trained locally on the privacy-preserving party using a small auxiliary dataset. The training process is conducted entirely in plaintext and does not involve shared collaboration among multiple parties. The privacy-preserving party performs backpropagation on the shadow model to obtain its gradient, which is then sent to the model layer selection module. The shadow model is used to estimate the true behavior of the attacker's model, assess the current level of data leakage, dynamically adjust the parameter layers in the model that need to be masked, and optimize the model layer selection strategy.

[0117] The core function of the shadow model is to simulate the model state of the attacker, helping the privacy protector to assess privacy risks in real time and make dynamic adjustments, thereby improving the effectiveness and flexibility of data label protection.

[0118] Step 4: Dynamically select the model layers that need to be masked based on the adjustable privacy budget:

[0119] The model layer selection module selects the model layers that need to be masked based on the privacy budget threshold. The core function of this module is to dynamically select the model layers that need to be masked based on the privacy budget threshold to optimize label privacy protection. The specific steps are as follows:

[0120] First, the shadow model The gradient values ​​calculated above are accumulated to the selection criterion parameter In the selection criteria parameter The gradient values ​​of each model layer are arranged in descending order, where the selection criterion parameter The model layer with a larger value is more important for protecting data labels.

[0121] Then, the privacy protection party uses the shadow model and auxiliary datasets, simulate the data label leakage degree of the current model locally. If the privacy leakage degree is higher than the set privacy budget threshold, it is necessary to further mask the model layer parameters to reduce the leakage risk. Therefore, the selection criterion parameters The model layer with the largest cumulative gradient value is selected as the most important layer for data label protection. After masking the parameters of the model layer, the privacy leakage degree of the masked model is re-evaluated. If the masked model still does not reach the set privacy budget threshold, continue to select the criterion parameters. The model layer with the larger gradient value is selected for masking. The process continues to iterate until the privacy leakage of the masked model label is lower than the privacy budget threshold set by the privacy protection party, or the criterion parameter is selected. There are no remaining model layers to select from.

[0122] After the iteration is completed, the privacy protection party will form a new ciphertext layer list with the selected model layers, and still use Indicates that the unselected model layers form a new list of layers to be reconstructed, and are still used Indicates that the updated new ciphertext layer list and the new ciphertext layer list Send it to the model layer masking module to start the next round of vertical federated model training.

[0123] The data label protection method of the present invention can effectively protect user data privacy in vertical federated learning and has at least the following advantages over existing methods:

[0124] First, this method uses secret sharing to mask parameters in important model layers, minimizing the risk of label privacy leakage. Secret sharing ensures that ciphertext and plaintext results remain consistent during linear operations, minimizing the impact on model accuracy while preserving privacy. This technical feature enables this method to achieve a good balance between privacy protection and model performance.

[0125] Secondly, this method uses a model layer selection module to dynamically identify and select the model layers most critical to label privacy protection, and only applies secret sharing to these layers. This selective masking mechanism effectively avoids masking all model layers, significantly reducing computational and communication overhead. Furthermore, by precisely selecting key layers, the method not only improves the efficiency of privacy protection but also enhances its practicality.

[0126] Furthermore, this method allows users to flexibly set the label privacy budget based on actual needs, providing a high degree of adjustability. When the privacy budget is high, only a small number of model layers need to be masked, which both meets privacy protection requirements and maximizes model performance. When the privacy budget is low, more model layers can be masked to provide stronger privacy protection. This flexibility enables this method to adapt to different application scenarios and user needs.

[0127] This invention effectively addresses the shortcomings of existing data labeling privacy protection methods, significantly improving privacy protection capabilities, model performance, and computational and communication efficiency. Leveraging secret sharing and a dynamic layer selection mechanism, this method effectively enhances data labeling privacy while ensuring model performance stability. By significantly reducing computational and communication overhead, its efficiency and practicality are further improved.

[0128] Finally, this method is simple in design, easy to implement, and applicable to a variety of vertical federated learning scenarios. Its efficient data privacy protection capabilities and flexible privacy budget settings make it promising for broad application in fields such as finance, healthcare, and smart cities that require data collaboration but place a high priority on privacy protection.

[0129] Example 2

[0130] This embodiment provides a data label protection method in vertical federated learning, and selects a neural network model with a three-layer structure, namely ; Set the number of participants in the vertical federated learning system to 2; set the privacy budget to 0.3. The specific implementation is as follows:

[0131] Parameter initialization:

[0132] First initialize the ciphertext layer list and the ciphertext layer list , the first layer model parameters Decomposed into two shared parameters through secret sharing method and , and distribute these two shared parameters to the two participants respectively. Finally, initialize the model layer selection criteria , so as to accumulate the gradient information of the model layer in the subsequent training process.

[0133] Step 1: Perform model-level masking based on secret sharing through the model-level masking module:

[0134] In the method initialization step, the first layer model parameters The masking has been completed through secret sharing. In order to prevent the attacker from inferring the real first-layer model parameters, the shared parameters Introducing parameter noise ,pass Calculate the noise sharing parameter 2 Since the second layer model parameters and the third layer model parameters Both belong to the plaintext layer, so there is no need to perform additional operations on the model parameters of these two layers.

[0135] Step 2, model security update process:

[0136] Secure forward propagation: Due to the first layer model parameters Secret sharing has been performed, so during forward propagation, the input data must first be secretly shared to obtain two input shared values, and then these two input shared values ​​are passed to the corresponding shared parameters respectively. and , calculate and get two output shared values, and finally, reconstruct the two output shared values ​​to get the second layer input data In plain text form, due to the second layer model parameters and the third layer model parameters Is the plaintext layer, the second layer input data Directly input the second-layer model parameters in sequence according to the traditional vertical federated learning method and the third layer model parameters , and obtain the final feature embedding of the model.

[0137] Safe back propagation: During the back propagation process, the loss value of the global model is calculated first, and the gradient calculation is started from the global output layer to obtain the gradient of the feature embedding. and the third layer model parameters It is a plaintext layer, which directly completes the back propagation of the gradient in plaintext to obtain the gradient of the first layer output. For the first layer parameters, the gradient is first decomposed into two shared gradient values ​​through secret sharing, and then the two shared gradient values ​​are input into the corresponding shared parameters respectively. and In , the gradients of all three-layer model parameters are calculated and the parameter updates are completed.

[0138] (4) Shadow model update process:

[0139] The privacy protection party first initializes the shadow model locally Its structure is the same as the attacker's model, but it is trained using an auxiliary dataset. During the training process, the gradient of the shadow model parameters is calculated through backpropagation, and the obtained gradient is passed to the model layer selection module to estimate the degree of privacy leakage of the current model.

[0140] (5) The model layer selection module selects the model layer to be masked based on the adjustable privacy budget:

[0141] The L1 norm of the gradient of each model layer parameter received in the shadow model is 0.6 ( ), 0.2( ), 0.4( ), where the first layer model parameters The gradient of the third layer model is the largest. The gradient of the second layer model is has the smallest gradient, so its importance priority is the lowest.

[0142] Since the first layer model parameters The masking is done at the method initialization, first simulating locally only the first layer model parameters The degree of label privacy leakage during masking. The results show that the degree of label leakage is 0.4, which is greater than the user-set privacy budget of 0.3. This indicates that the current masking strategy cannot meet the privacy protection requirements, and therefore it is necessary to introduce more model layers for masking.

[0143] Then, sort the model parameters in descending order of the cumulative gradient values, starting from the third layer with the second largest gradient value. First, add it to the ciphertext layer list and simulate locally while adjusting the first layer model parameters and the third layer model parameters The degree of label privacy leakage during masking. Simulation results show that the label leakage degree at this time is 0.2, which is less than the user-set privacy budget of 0.3, so the selection process of important model layers can be terminated.

[0144] Finally, the new ciphertext layer list is updated to The remaining unselected model layers form a new list of layers to be reconstructed , and update the new ciphertext layer list and the list of new layers to be refactored The data is sent to the model layer masking module to start the next round of vertical federated model training.

[0145] The method of this embodiment can maximize model performance while ensuring data privacy, solving the problem that existing technologies find it difficult to balance the two. This method has low computational and communication overheads and can operate efficiently in practical application scenarios, thus having good feasibility. The method also provides users with an adjustable privacy budget option, allowing users to flexibly adjust the privacy budget value based on their own privacy needs and computing resources to achieve different levels of privacy protection.

[0146] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0147] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims. The information disclosed in the background technology section of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.

Claims

1. A data label protection method in vertical federated learning, characterized by: Before training the model, the privacy protector initializes the ciphertext layer list, the list of layers to be reconstructed, and the selection criteria parameters, and then starts training, including: Step 1: Perform secret sharing masking on the corresponding model layer parameters in the ciphertext layer list; Perform secret reconstruction on the corresponding model layer parameters in the list of layers to be reconstructed; Step 2: Model security update: Secure forward propagation: If the model layer is a ciphertext layer, secret sharing and secret reconstruction are performed on the input to obtain the input for the next model layer. The above process is iterated until the final feature embedding of the output model is obtained. Secure backpropagation: Calculate the global model loss and perform backpropagation from the global output layer of the global model. If the model layer is a ciphertext layer, perform secret sharing and secret reconstruction on the input gradient value to obtain the plaintext gradient value of the previous model layer. Iterate the above process until the parameters of each layer of the model are updated. Step 3, shadow model update: The privacy protector builds and trains the shadow model locally, calculates the gradient value of the shadow model, and accumulates the gradient value of the shadow model into the selection criterion parameter; Step 4: Select and adjust the model layers to be masked according to the preset privacy budget threshold and the cumulative gradient value in the selection criterion parameter, and update the new ciphertext layer list and the new to-be-reconstructed layer list; Step 5: Return to step 1 with the new ciphertext layer list and the new layer list to be reconstructed to start the next round of training until the training is completed.

2. The data label protection method in vertical federated learning according to claim 1 is characterized in that: In step 1, the corresponding model layer parameters in the ciphertext layer list are masked for secret sharing in the following manner: Decompose the model layer parameters corresponding to the ciphertext layer list into shared parameter 1 and shared parameter 2 through secret sharing, and add noise to shared parameter 2 to obtain noisy shared parameter 2; In step 1, the complete model layer parameters corresponding to the reconstructed layer list are not secretly reconstructed, and the incomplete model layer parameters corresponding to the reconstructed layer list are secretly reconstructed in the following manner, including: By summing, a noise parameter is introduced into the shared parameter 2 to obtain the noisy shared parameter 2, and the shared parameter 1 and the noisy shared parameter 2 are summed to reconstruct the model layer parameters; In the secure forward propagation of step 2, if the model layer is a plaintext layer, the forward propagation method of traditional vertical federated learning is used for calculation; In the secure backpropagation of step 2, if the model layer is a plaintext layer, the backpropagation method of traditional vertical federated learning is used for calculation; In step 4, the model layers to be masked are selected and adjusted according to the preset privacy budget threshold in the following manner, and a new ciphertext layer list and a new list of layers to be reconstructed are updated, including: The privacy protector locally simulates the degree of data label leakage of the current model through a shadow model. Based on the degree of data label leakage and the privacy budget threshold preset by the privacy protector, the privacy protector adds a model layer for parameter masking so that the degree of data label leakage of the masked model is less than the privacy budget threshold or all parameters of the model layers are masked. The privacy protector updates the new ciphertext layer list and the new list of layers to be reconstructed according to the adjusted masked model layer.

3. The data label protection method in vertical federated learning according to claim 1 or 2, characterized in that: Initialize the ciphertext layer list, the to-be-reconstructed layer list, and the selection criteria parameters as follows: The initialized ciphertext layer list is , Indicates the model layer in the model trained by the privacy protection party that requires secret sharing for masking; The initialized list of layers to be reconstructed is , represents the model layer that needs to be secretly reconstructed in the privacy-preserving party’s trained model, where is the total number of model layers; Initialization selection criterion parameters , in subsequent training, it is used to continuously accumulate the gradient value of the model to the selection criterion parameter middle.

4. The data label protection method in vertical federated learning according to claim 3 is characterized in that: In step 1, the ciphertext layer list is received by the model layer masking module set by the privacy protection party , for the ciphertext layer list The corresponding model layer parameters in the secret sharing masking process include: The model layer masking module uses secret sharing to mask the ciphertext layer list. Elements The corresponding model Layer Parameters Decompose into shared parameters and shared parameter 2 , by summing Shared parameter 2 Introducing noise parameters , get the noise-added shared parameter 2; In step 2, the model layer masking module receives the list of layers to be reconstructed , treat the reconstruction layer list The corresponding model layer parameters in are secretly reconstructed, including: The model layer masking module determines the list of layers to be reconstructed Elements The corresponding model Layer Parameters Is it complete? If it is complete, no processing is performed. Otherwise, the layer parameters are secretly reconstructed as follows: by summing Shared parameter 2 Noise parameters are introduced , get the noise sharing parameter 2 , sum the shared parameter 1 and the noise-added shared parameter 2 , reconstruct the plaintext model Layer Parameters .

5. The data label protection method in vertical federated learning according to claim 4 is characterized in that: In the secure forward propagation of step 2, if the model layer is a ciphertext layer, the input is secret-shared and secret-reconstructed in the following manner to obtain the input for the next model layer, including: Decompose the input of the model layer into two shared input values ​​through secret sharing. Input each shared input value into the corresponding shared parameter of the model layer to obtain the corresponding two shared output values ​​of the model layer. Secretly reconstruct the two shared output values ​​in the following manner to obtain the input of the next model layer. The way to secretly reconstruct the two shared output values ​​to get the input of the next model layer is: Add noise to the shared output value 2 corresponding to the shared parameter 2 of the model layer to obtain the noisy shared output value 2, and sum the shared output value 1 corresponding to the shared parameter 1 of the model layer and the noisy shared output value 2 to obtain the input of the next model layer; In the secure backpropagation of step 2, if the model layer is a ciphertext layer, the plaintext gradient value of the previous model layer is obtained by performing secret sharing and secret reconstruction on the input gradient value in the following manner, including: The input gradient value of the model layer is decomposed into two shared gradient values ​​through secret sharing. The two shared gradient values ​​are respectively input into the corresponding shared parameters of the model layer to obtain the corresponding two shared gradient values ​​of the previous layer of the model layer. The two shared gradient values ​​of the previous layer are secretly reconstructed in the following way to obtain the plaintext gradient value of the previous model layer; The method of secretly reconstructing the two shared gradient values ​​of the previous layer to obtain the plaintext gradient value of the previous model layer is: Add noise to the previous layer shared gradient value two corresponding to the shared parameter two of the model layer to obtain the noisy previous layer shared gradient value two, and sum the previous layer shared gradient value one corresponding to the shared parameter one of the model layer and the noisy previous layer shared gradient value two to obtain the plaintext previous model layer gradient value.

6. The data label protection method in vertical federated learning according to claim 5 is characterized in that: In step 3, the privacy protector constructs and trains the shadow model locally and calculates the gradient value of the shadow model in the following manner, including: The privacy-preserving party constructs a shadow model locally with the same structure as the attacker's model but with different parameters. And use the auxiliary data set to train the shadow model locally in plain text. After training, perform back propagation calculation on the shadow model to obtain the shadow model The gradient value of .

7. The data label protection method in vertical federated learning according to claim 6 is characterized in that: In step 4, the privacy protection party selects and adjusts the model layer to be masked according to the preset privacy budget threshold through the model layer selection module in the following manner, including: The privacy protection side selects the shadow model from the model layer The gradient value is accumulated to the selection criterion parameter In the selection criteria parameter The accumulated gradient values ​​of each model layer in are arranged in descending order; Privacy protection is achieved by selecting modules at the model layer through shadow models and auxiliary data sets, simulate the data label leakage degree of the current model locally, if the data label leakage degree is judged to be higher than the privacy budget threshold set by the privacy protection party, then select the criterion parameters from the descending order The parameters of the model layer with the largest cumulative gradient value are selected for masking, and the above process is repeated iteratively until the degree of leakage of the masked model data label is less than the set privacy budget threshold or the selection criterion parameter There are no model layers to choose from.

8. The data label protection method in vertical federated learning according to claim 7 is characterized in that: In step 4, the privacy protector updates the new ciphertext layer list and the new to-be-reconstructed layer list in the following manner, including: The privacy protection party will select the model layers that need to be masked to form a new ciphertext layer list , the model layers that do not need masking are not selected to form a new list of layers to be reconstructed , the new ciphertext layer list and the new list of layers to be refactored Passed to the model-level masking module for model-level masking processing for secret sharing.

9. A processing device comprising: at least one memory for storing one or more programs; At least one processor is capable of executing one or more programs stored in the memory, and when the one or more programs are executed by the processor, the processor is capable of implementing the method according to any one of claims 1 to 8.

10. A readable storage medium storing a computer program, which can implement the method according to any one of claims 1 to 8 when the computer program is executed by a processor.

Images