A Training Method and System for Vertical Federated GRU Models Based on Secret Sharing and Differential Privacy

Abstract

This specification provides a method and system for training a vertical federated GRU model based on secret sharing and differential privacy. The method includes: determining encrypted sample data; splitting the encrypted sample data to obtain multiple encrypted sample data fragments; inputting the multiple encrypted sample data fragments into a first GRU model to obtain prediction result fragments corresponding to the multiple encrypted sample data fragments, wherein the first GRU model is deployed on participating nodes; calculating the initial model gradient of the first GRU model based on the prediction result fragments and the encrypted label data fragments sent by the central node; obtaining a target noise vector; adding the target noise vector to the initial model gradient to obtain an updated model gradient; and sending the updated model gradient to the central node so that the central node updates a second GRU model based on the updated model gradient, wherein the second GRU model is deployed on the central node.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a training method and system for a vertical federated GRU model based on secret sharing and differential privacy. Background Technology

[0002] With the development of fields such as the Internet of Things, fintech, and healthcare, the demand for time-series data analysis and modeling has increased dramatically. Gated Recurrent Units (GRUs) have become one of the mainstream models for processing time-series data due to their high parameter efficiency and fast training speed. However, in practical applications, time-series data is often scattered among different stakeholders. Considering data privacy and security issues, some data cannot be directly shared centrally, thus forming "data silos." For example, banks hold users' transaction records, while e-commerce platforms hold users' shopping behavior data; both need to jointly model data to improve the effectiveness of credit scoring models.

[0003] Based on this, the federated learning strategy for models has emerged. The core idea of ​​this strategy is that multiple participants collaboratively train a global model using their own data without sharing the original data. However, when performing federated learning on GRU, because participants may hold sensitive data (such as medical records and transaction records), data leakage needs to be prevented. Current data encryption methods applied to the parameter calculation process of GRU require significant computational resources, leading to low efficiency in federated learning. Therefore, an effective solution is urgently needed that can protect data privacy and security while ensuring the efficiency of federated learning for GRU. Summary of the Invention

[0004] In view of this, embodiments of this specification provide two methods for training vertical federated GRU models based on secret sharing and differential privacy. One or more embodiments of this specification also relate to two apparatuses for training vertical federated GRU models based on secret sharing and differential privacy, a system for training vertical federated GRU models based on secret sharing and differential privacy, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a method for training a vertical federated GRU model based on secret sharing and differential privacy is provided, applied to participating nodes, including:

[0006] Determine the encrypted sample data, split the encrypted sample data, and obtain multiple encrypted sample data fragments;

[0007] The multiple encrypted sample data shards are input into the first GRU model to obtain the prediction result shards corresponding to the multiple encrypted sample data shards, wherein the first GRU model is deployed on the participating nodes;

[0008] Based on the predicted result fragments and the encrypted tag data fragments sent by the central node, calculate the initial model gradient of the first GRU model;

[0009] A target noise vector is obtained, and the target noise vector is added to the initial model gradient to obtain an updated model gradient. The updated model gradient is then sent to the central node so that the central node updates the second GRU model according to the updated model gradient. The second GRU model is deployed on the central node.

[0010] According to a second aspect of the embodiments of this specification, a training apparatus for a vertical federated GRU model based on secret sharing and differential privacy is provided, applied to participating nodes, including:

[0011] The splitting module is configured to determine the encrypted sample data, split the encrypted sample data, and obtain multiple encrypted sample data fragments.

[0012] The input module is configured to input the plurality of encrypted sample data shards into a first GRU model to obtain prediction result shards corresponding to the plurality of encrypted sample data shards, wherein the first GRU model is deployed on the participating node;

[0013] The calculation module is configured to calculate the initial model gradient of the first GRU model based on the fragmentation of the prediction result and the fragmentation of the encrypted tag data sent by the central node.

[0014] The sending module is configured to acquire a target noise vector, add the target noise vector to the initial model gradient to obtain an updated model gradient, and send the updated model gradient to the central node so that the central node updates the second GRU model according to the updated model gradient, wherein the second GRU model is deployed on the central node.

[0015] According to a third aspect of the embodiments of this specification, another method for training a vertical federated GRU model based on secret sharing and differential privacy is provided, applied to a central node, including:

[0016] Determine the encrypted tag data, and split the encrypted tag data to obtain multiple encrypted tag data fragments;

[0017] The multiple encrypted tag data fragments are sent to the participating nodes, and the updated model gradient of the first GRU model sent by the participating nodes is received. The first GRU model is deployed on the participating nodes, and the updated model gradient is obtained by adding a target noise vector to the initial model gradient. The initial model gradient is calculated based on the multiple encrypted tag data fragments.

[0018] The second GRU model is updated according to the updated model gradient to obtain the updated second GRU model, and the updated model parameters of the updated second GRU model are sent to the participating nodes, wherein the second GRU model is deployed at the central node.

[0019] According to a fourth aspect of the embodiments of this specification, another training apparatus for a vertical federated GRU model based on secret sharing and differential privacy is provided, applied to a central node, comprising:

[0020] The splitting module is configured to determine the encrypted tag data, split the encrypted tag data, and obtain multiple encrypted tag data fragments;

[0021] The communication module is configured to send the multiple encrypted tag data fragments to the participating nodes and receive the updated model gradient of the first GRU model sent by the participating nodes, wherein the first GRU model is deployed on the participating nodes, and the updated model gradient is obtained by adding a target noise vector to the initial model gradient, and the initial model gradient is calculated based on the multiple encrypted tag data fragments.

[0022] The update module is configured to update the second GRU model according to the update model gradient, obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed on the central node.

[0023] According to a fifth aspect of the embodiments of this specification, a vertical federated GRU model training system based on secret sharing and differential privacy is provided, including participating nodes and a central node, wherein,

[0024] The central node is configured to determine encrypted tag data, split the encrypted tag data to obtain multiple encrypted tag data fragments, and send the multiple encrypted tag data fragments to the participating nodes;

[0025] The participating node is configured to: determine encrypted sample data; split the encrypted sample data to obtain multiple encrypted sample data fragments; input the multiple encrypted sample data fragments into a first GRU model to obtain prediction result fragments corresponding to the multiple encrypted sample data fragments; calculate the initial model gradient of the first GRU model based on the prediction result fragments and the encrypted label data fragments sent by the central node; obtain a target noise vector; add the target noise vector to the initial model gradient to obtain an updated model gradient; and send the updated model gradient to the central node, wherein the first GRU model is deployed on the participating node.

[0026] The central node is configured to update the second GRU model according to the updated model gradient, obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed on the central node.

[0027] According to a sixth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0028] Memory and processor;

[0029] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above method.

[0030] According to a seventh aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0031] According to an eighth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0032] This specification provides an embodiment of a vertical federated GRU model training method based on secret sharing and differential privacy, applied to participating nodes, comprising: determining encrypted sample data; splitting the encrypted sample data to obtain multiple encrypted sample data fragments; inputting the multiple encrypted sample data fragments into a first GRU model to obtain prediction result fragments corresponding to the multiple encrypted sample data fragments, wherein the first GRU model is deployed on the participating nodes; calculating the initial model gradient of the first GRU model based on the prediction result fragments and encrypted label data fragments sent by the central node; obtaining a target noise vector; adding the target noise vector to the initial model gradient to obtain an updated model gradient; and sending the updated model gradient to the central node so that the central node updates a second GRU model based on the updated model gradient, wherein the second GRU model is deployed on the central node.

[0033] In the above method, when performing federated training on the GRU model, the encrypted sample data can be split into multiple encrypted sample data slices, making it impossible for any party to reconstruct the original encrypted sample data from a single encrypted sample data slice. These multiple encrypted sample data slices are input into the first GRU model to obtain prediction result slices corresponding to each slice. Based on these prediction result slices and the encrypted label data slices sent by the central node, the initial model gradient of the first GRU model is calculated, and the target noise vector is obtained. This target noise vector is added to the initial model gradient to obtain the updated model gradient, thus achieving secondary encryption of the model gradient. This secondary encrypted updated model gradient is then sent to the central node, allowing the central node to update the second GRU model based on the updated model gradient, enabling subsequent federated training of the second GRU model. By bringing the sample data encryption process forward, the parameter calculation process in the first GRU model is directly performed based on the encrypted sample data slices, avoiding the large amount of computational resources consumed by encrypting intermediate results. This improves the federated learning efficiency of the GRU model and ensures the privacy and security of the training data through secondary encryption. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating a training method for a vertical federated GRU model based on secret sharing and differential privacy, provided in one embodiment of this specification.

[0035] Figure 2 This is a schematic diagram illustrating the process of applying a vertical federated GRU model training method based on secret sharing and differential privacy to participating nodes and a central node, as provided in one embodiment of this specification.

[0036] Figure 3This is a flowchart of another method for training a vertical federated GRU model based on secret sharing and differential privacy, provided in one embodiment of this specification.

[0037] Figure 4 This is a flowchart illustrating the processing steps of a vertical federated GRU model training method based on secret sharing and differential privacy, provided in one embodiment of this specification.

[0038] Figure 5 This is a schematic diagram of the structure of a vertical federated GRU model training device based on secret sharing and differential privacy, provided in one embodiment of this specification.

[0039] Figure 6 This is a schematic diagram of another vertical federated GRU model training device based on secret sharing and differential privacy provided in one embodiment of this specification;

[0040] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0041] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0042] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0044] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0045] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0046] GRU (Gated Recurrent Unit) is an improved recurrent neural network (RNN) architecture for processing temporal data. It alleviates the vanishing gradient problem of traditional RNNs by introducing a "gating mechanism," enabling the model to better capture long-term dependencies. It uses two gates to control the flow of information: Update Gate: determines how much historical information is retained in the current state; Reset Gate: determines whether to ignore previous hidden states.

[0047] RNN: Recurrent Neural Network, is a neural network structure designed specifically for processing sequential data, suitable for time-dependent tasks such as text, speech, or time series modeling.

[0048] MPC: Multi-Party Computation, a branch of cryptography, refers to the technique of multiple parties collaboratively computing the result of a function without revealing their private inputs. It is widely used in the field of privacy-preserving computing.

[0049] SS protocol: Secret Sharing, also known as secret sharing, is a commonly used technique in MPC. Its core idea is to divide a secret value into multiple shares. Only when enough shares are combined can the secret be recovered; otherwise, no information can be obtained.

[0050] Differential privacy is a mathematically defined privacy framework designed to ensure the security of personal data during data analysis and machine learning. Its core idea is to incorporate appropriate noise into query results or model training to protect individual data from being leaked, while still allowing useful information to be extracted from the overall dataset.

[0051] In practical applications, the demand for time-series data analysis and modeling has grown dramatically with the development of fields such as the Internet of Things, fintech, and healthcare. Gated Recurrent Units (GRUs) have become one of the mainstream models for processing time-series data due to their high parameter efficiency and fast training speed. However, in real-world applications, time-series data is often scattered across different institutions or stakeholders, and due to data privacy and security regulations, data cannot be directly and centrally shared, thus forming "data silos." For example, banks hold users' transaction records, while e-commerce platforms hold users' shopping behavior data; both need to be jointly modeled to improve the effectiveness of credit scoring models.

[0052] Federated learning allows multiple participants to collaboratively train a model without sharing the original data. It can be categorized into horizontal federated learning, vertical federated learning, and federated transfer learning. Vertical federated learning involves participants having the same samples but different feature spaces. However, it still faces the following challenges: Privacy issues: Participants in model training may possess sensitive data (such as medical records or transaction records), requiring measures to prevent data leakage. Gradient propagation may be reverse-engineered to extract training data. High computational complexity: The activation function of GRU involves non-linear operations, and existing secure computation methods used in deep learning (such as isomorphic encryption) are inefficient.

[0053] Specifically, current vertical federated learning methods rely on logistic regression or tree models, which cannot capture the dynamic characteristics of time-series data; while federated GRU schemes only support horizontal data partitioning and cannot adapt to vertical scenarios where financial institutions and third-party data platforms have complementary features. Furthermore, traditional homomorphic encryption protocols are technically inefficient in GRU gating parameter aggregation, and further exploration is needed for more efficient algorithms. The main issues are as follows: High overhead of nonlinear operations: GRU activation functions (such as sigmoid and tanh) require high-order polynomial approximation (such as Taylor expansion) under homomorphic encryption, which leads to excessively long computation time for a single activation and introduces significant approximation errors; Defects of combining vertical federated learning with GRU: Current vertical federated learning cannot handle time-series data, while current federated GRU schemes only support horizontal partitioning and do not address the scenario of feature distribution across institutions in the vertical direction; Explosion of communication rounds: Gating computation at each time step requires multiple ciphertext multiplication interactions. For example, the computation of an activation function requires multiple rounds of communication, leading to an exponential increase in communication overhead for long sequence training; Insufficient secret sharing (SS): Current SS protocols have communication bottlenecks (such as multiple rounds of interaction per iteration) or accuracy loss (such as gradient bias caused by noise) in GRU gating computation.

[0054] This specification provides two training methods for vertical federated GRU models based on secret sharing and differential privacy. It also relates to two training devices for vertical federated GRU models based on secret sharing and differential privacy, a training system for vertical federated GRU models based on secret sharing and differential privacy, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0055] See Figure 1 , Figure 1 The flowchart illustrates a training method for a longitudinal federated GRU model based on secret sharing and differential privacy, according to an embodiment of this specification, applied to participating nodes, and specifically includes the following steps.

[0056] Step 102: Determine the encrypted sample data, split the encrypted sample data, and obtain multiple encrypted sample data fragments.

[0057] Specifically, the vertical federated GRU model training method based on secret sharing and differential privacy provided in the embodiments of this specification can be applied to the vertical federated training of GRU models. Vertical federated learning is a distributed machine learning method suitable for situations where different data holders have little overlap in the sample dimension but a lot of overlap in the feature dimension. For example, two different companies may have some of the same customers, but collect different types of data (one may have the customer's transaction records, while the other may have information about the customer's social media activities). Vertical federated learning allows these entities to jointly train the model without sharing the original data, thereby protecting data privacy. GRU (Gated Recurrent Unit) is a variant of Recurrent Neural Network (RNN) specifically designed for processing sequential data. It effectively solves the gradient vanishing problem in traditional RNNs by introducing update gates and reset gates to control the information flow, enabling the model to capture long-term dependencies. GRU is widely used in natural language processing, time series prediction, and other fields. The vertical federated GRU model combines the two; a vertical federated GRU model refers to a GRU model trained under the vertical federated learning framework. This setup is particularly suitable for scenarios that require processing sequential data and involve multi-party collaboration. Specifically, suppose multiple institutions want to jointly train a time-series-based predictive model, but they each possess different attribute data for users or objects. For example, in healthcare, Hospital A might have patients' electronic health records, while Hospital B might have patients' genomic data. A longitudinal federated GRU model can be used to predict disease progression while ensuring the security of sensitive information.

[0058] In practical applications, each participating node uploads intermediate results (such as gradients) computed locally, rather than the raw data. These intermediate results are typically encrypted to further protect privacy. All participating nodes jointly maintain a global GRU model. In each iteration, each participating node computes its local gradient based on its own data and sends it to the central node. The central node aggregates these local gradients, updates the global model parameters, and then distributes the updated parameters back to the participating nodes. Throughout this process, since only encrypted gradients are exchanged, no party can directly access the data of others, thus effectively protecting the data privacy of all parties. By avoiding direct data sharing, the risk of data leakage is significantly reduced. Integrating features from different sources allows for the construction of more comprehensive and accurate models. This helps meet increasingly stringent privacy requirements.

[0059] Encrypted sample data can be understood as encrypted sample data held and provided by participating nodes. Sample data can be, for example, user transaction record data held by banks or user electronic medical record data held by hospitals.

[0060] Specifically, the sample data held and provided by participating nodes can be encrypted to obtain encrypted sample data. This encrypted sample data can then be split into multiple encrypted sample data fragments, ensuring that the sample data is encrypted and stored in a distributed manner, making it impossible for any single party to recover the original sample data.

[0061] In practical applications, sample data can be time-series data, which refers to a series of observations recorded in chronological order. Each observation is associated with a specific point in time or time period. This type of data is widely found in many fields, such as finance, meteorology, healthcare, and the Internet of Things. Understanding time-series data is crucial for analyzing trends, predicting future events, and making data-driven decisions. The basic characteristics of time-series data include: Time dependence: A key characteristic of time-series data is its sequential nature and temporal continuity. This means that there is some form of dependency between data points, and current values ​​are often influenced by past values. Timestamps: Each record has a clear time identifier, which may be a specific point in time (e.g., 2023-06-06 09:21) or a fixed time interval (e.g., the first day of each day or month). Periodicity / Seasonality: Many time-series data exhibit periodic patterns, such as daytime temperatures varying with the night, and stock market fluctuations showing weekly and annual patterns. Trends: In the long term, time-series data may show an upward or downward trend, reflecting the continuous impact of internal or external factors on the data. Noise: Actual time series data often contains random fluctuations or noise, which are short-term variations caused by unpredictable factors.

[0062] The GRU model trained using the longitudinal federated GRU model training method based on secret sharing and differential privacy provided in this specification can be applied to time series data analysis scenarios. For example, it can be applied to financial market analysis: stock prices, exchange rates, and interest rates are typical time series data, and analyzing historical data can help investors make buying and selling decisions; sales forecasting: retailers can use past sales records to predict future sales, thereby optimizing inventory management and marketing strategies; environmental monitoring: weather stations collect data such as temperature, humidity, and wind speed for weather forecasting and climate research; industrial monitoring: sensors in manufacturing plants continuously record equipment status parameters (such as temperature and pressure) to enable preventative maintenance; and health monitoring: wearable devices record information such as heart rate and steps, which is helpful for personal health management and medical research. The embodiments in this specification do not limit this application.

[0063] In specific implementation, determining the encrypted sample data includes:

[0064] Obtain the sample data stored in the participating nodes;

[0065] The sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

[0066] In practical applications, the preset encryption algorithm can be a symmetric encryption algorithm, an asymmetric encryption algorithm, a stream encryption algorithm, a homomorphic encryption algorithm, a timestamp encryption algorithm, or a Beaver triplet encryption algorithm, etc. Alternatively, the sample data can be encrypted according to the MPC protocol parameters negotiated between the participating nodes and the central node. This specification does not limit this aspect in the embodiments.

[0067] Specifically, Beaver triples can be understood as a technique in secure multi-party computation (MPC) used to perform secure two- or multi-party computations without exposing the private inputs of participating nodes. A Beaver triple can be used to implement secure multiplication operations. A Beaver triple consists of three random numbers a, b, and c, where c = a × b itself.

[0068] In summary, by encrypting the sample data, encrypted sample data is obtained, achieving initial encryption of the sample data. This eliminates the need for encryption during the intermediate calculations of the GRU model, thereby reducing the consumption of computing resources.

[0069] Furthermore, after obtaining the sample data stored in the participating nodes, the process further includes:

[0070] The sample data is standardized to obtain standardized sample data;

[0071] The step of encrypting the sample data according to a preset encryption algorithm to obtain encrypted sample data includes:

[0072] The standardized sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

[0073] Specifically, the sample data can be standardized to obtain standardized sample data, and then encrypted according to a preset encryption algorithm to obtain encrypted sample data. Subsequently, the encrypted sample data can be split to obtain multiple encrypted sample data fragments.

[0074] In practical applications, the sample data can be local time-series data of the participating nodes, such as a time-series feature matrix. Then, the time-series data can be standardized, such as by performing Z-score standardization to obtain standardized sample data. The standardized sample data can be encrypted and split into multiple encrypted sample data shards (i.e., secret sharing shards), and the multiple encrypted sample data shards can be distributed and stored in the participating nodes.

[0075] In summary, standardizing the sample data improves its quality, making the subsequent GRU model training process more efficient, accelerating training, reducing the number of iterations required for convergence, and increasing model accuracy. Furthermore, encrypting and distributing the sample data ensures that no participating node or central node can independently reconstruct the original sample data, thus guaranteeing data privacy and security.

[0076] Step 104: Input the multiple encrypted sample data shards into the first GRU model to obtain the prediction result shards corresponding to the multiple encrypted sample data shards, wherein the first GRU model is deployed on the participating node.

[0077] The first GRU model can be understood as a GRU model deployed locally on the participating node. This first GRU can be used to calculate the encrypted sample data provided by the participating node, obtain the prediction result shards, and be used to calculate the model gradient of the first GRU model in the subsequent calculation.

[0078] Specifically, multiple encrypted sample data slices can be input into the first GRU model, and calculations can be performed in the first GRU model to obtain prediction result slices corresponding to the multiple encrypted sample data slices. Furthermore, there can be multiple prediction result slices, so each encrypted sample data slice corresponds to one prediction result slice.

[0079] In practical applications, during the joint training of the GRU model, the chanting state at the current time step and the prediction result slices output at the current time step can be calculated during the forward propagation process. The specific implementation is as follows: Inputting the multiple encrypted sample data slices into the first GRU model to obtain the prediction result slices corresponding to the multiple encrypted sample data slices includes:

[0080] The multiple encrypted sample data are fragmented and input into the first GRU model. Based on the multiple encrypted sample data, the polynomial corresponding to the first activation function, and the hidden state of the previous time step, the update gate and reset gate of the first GRU model are calculated.

[0081] Based on the reset gate, the polynomial corresponding to the second activation function, and the hidden state of the previous time step, calculate the candidate hidden state corresponding to the first GRU model;

[0082] Calculate the hidden state at the current time step based on the update gate and the candidate hidden states;

[0083] Based on the hidden state of the current time step, calculate and output the prediction result fragments corresponding to the multiple encrypted sample data fragments.

[0084] The GRU model effectively manages the memory and forgetting processes by introducing update and reset gates to control the information flow. The update and reset gates can be understood as update gate vectors and reset gate vectors, respectively. The update gate determines the extent to which new information at the current moment should be used to update the hidden state. It uses a value between 0 and 1 to weight the ratio of old to new information. The update gate helps the GRU model decide how much old information to retain and how much new input to accept. If the update gate is close to 1, more old information is retained; if the update gate is close to 0, more new information is used.

[0085] The reset gate determines how much old information needs to be ignored when calculating new candidate hidden states. It can be seen as a filter used to selectively discard some historical information. A lower reset gate indicates that the current time step is more inclined to ignore some old information in order to better adapt to new input information; conversely, a higher reset gate indicates that the current time step places more emphasis on previous states.

[0086] The hidden state is one of the core outputs of the GRU model. It contains all the sequence information processed by the GRU model. At each time step, the hidden state is adjusted according to the guidance of the update gate and the reset gate. The hidden state can be regarded as the memory unit of GRU, storing all relevant information of the GRU model from the beginning to the current time step.

[0087] Both the first and second activation functions are gating parameters of the GRU model. The first activation function can be a sigmoid function, and the second activation function can be a tanh function. The polynomial corresponding to the first activation function can be an approximate representation of the first activation function, and the polynomial corresponding to the second activation function can be an approximate representation of the second activation function.

[0088] Specifically, after inputting multiple encrypted sample data slices into the first GRU model, the update gate and reset gate of the first GRU model can be calculated based on the multiple encrypted sample data, the polynomial corresponding to the sigmoid function, and the hidden state of the previous time step. In the initial calculation, the hidden state of the previous time step can be the initial hidden state (i.e., an all-zero vector). Based on the reset gate, the polynomial corresponding to the tanh function, and the hidden state of the previous time step, the candidate hidden state corresponding to the first GRU model is calculated. Based on the update gate and the candidate hidden state, the hidden state of the current time step is calculated. Finally, based on the hidden state of the current time step, the prediction result slices corresponding to the multiple encrypted sample data slices are calculated and output.

[0089] In practical applications, during the forward propagation phase of the jointly trained GRU model, each participating node can set its initial hidden state to an all-zero vector and store it using a secret sharing mechanism. This means the initial hidden state can be encrypted and fragmented, resulting in a distributed storage of the encrypted initial hidden state. Each participating node calculates the update and reset gates using secure matrix multiplication based on its current hidden state (i.e., the hidden state of the previous time step, which becomes the initial hidden state during the first computation) and multiple encrypted sample data fragments. The reset gate and the current hidden state are then combined to calculate a candidate hidden state. Finally, the update gate is used to mix this candidate hidden state with the hidden state of the previous time step to obtain a new hidden state (i.e., the hidden state of the current time step). During the backpropagation phase of the jointly trained GRU model, each participating node can perform a linear transformation on its current hidden state at time step t to obtain fragments of the prediction results.

[0090] Specifically, the update gate can be calculated using the following formula.

[0091]

[0092] Where t is the current time step, It is a fragment of the currently input encrypted sample data. It is the hidden state of the previous time step. Let be the polynomial corresponding to the first activation function. It's an update gate. To update the gate's bias term, It is the weight matrix of the updated gate.

[0093] The reset door can be calculated using the following formula.

[0094]

[0095] in, It's a door reset. It is the weight matrix of the reset gate. It resets the door's bias.

[0096] The candidate hidden state can be calculated using the following formula.

[0097]

[0098] in, It is a candidate hidden state. It is the polynomial corresponding to the second activation function. It is the weight of Tachibana in the hidden state. It is the bias term of the hidden state.

[0099] The hidden state at the current time step can be calculated using the following formula. .

[0100]

[0101] The predicted result fragments can be calculated using the following formula. .

[0102]

[0103] in, It is the bias term of the prediction result output by the first GRU model.

[0104] In summary, by using a secret sharing protocol to perform computation on encrypted sample data fragments, privacy-preserving collaborative computation of training data for each participating node is achieved, ensuring that the original data of each participating node is not leaked during the training process, thereby guaranteeing data privacy and security.

[0105] Furthermore, before calculating the update gate and reset gate of the first GRU model based on the multiple encrypted sample data, the polynomial corresponding to the first activation function, and the hidden state of the previous time step, the method further includes:

[0106] The first activation function is approximated as the polynomial corresponding to the first activation function;

[0107] The second activation function can be approximated as the polynomial corresponding to the second activation function.

[0108] Specifically, the first activation function can be approximated by decomposing it into multiple low-order polynomial terms to obtain the low-order polynomial corresponding to the first activation function; correspondingly, the second activation function can be approximated by decomposing it into multiple low-order polynomial terms to obtain the low-order polynomial corresponding to the second activation function.

[0109] In practical applications, the polynomial corresponding to the first activation function is shown in the following formula.

[0110]

[0111] The polynomial corresponding to the second activation function is shown in the following formula.

[0112]

[0113] In summary, by decomposing the first and second activation functions of the GRU model into multiple low-order polynomial terms and performing distributed computation on each participating node through a secret sharing protocol, the computational complexity of the GRU model's gating function is reduced from the polynomial level to the linear level, making it particularly suitable for long-sequence data scenarios. In terms of model performance, the gating structure and temporal modeling capabilities of GRU are fully preserved, resulting in better performance compared to replacing GRU with a linear model.

[0114] Step 106: Calculate the initial model gradient of the first GRU model based on the predicted result fragments and the encrypted tag data fragments sent by the central node.

[0115] In this context, the central node can be understood as a participant in the federated training of the GRU model. This central node can hold labeled data, such as default labels in financial risk control or disease labels in medical prediction. Furthermore, this central node can also function as a central server, where the aggregation of model gradients sent by various participating nodes and the training of the GRU model can be performed.

[0116] Specifically, the central node can encrypt the label data it holds to obtain encrypted label data, and then split the encrypted label data into multiple encrypted label data fragments, which are then sent to each participating node. Each participating node can calculate the initial model gradient of the first GRU model deployed locally on its own system based on its own calculated prediction result fragments and the multiple encrypted label data fragments sent by the central node. It is understandable that the process of the central node encrypting and splitting the label data is similar to the process of the participating nodes encrypting and splitting the sample data described above, and this embodiment will not repeat the details here.

[0117] In practical applications, the central node can negotiate MPC protocol parameters with at least one participating node. These MPC protocol parameters may include parameters such as the finite field size and random number seed, which can be used for data encryption and data transmission. (Setting the participant set is then required.) ,in The central node holds the tagged data, and the others are participating nodes. The parties negotiate MPC protocol parameters (such as the size of the secretly shared finite field and the random number seed), and process the local time-series data. Standardize and generate a secret sharing value. > (i.e., encrypted sample data fragments or encrypted tag data fragments), where... This represents a sample or feature matrix of time series data, where the dimension of the time series data is T×di.

[0118] In specific implementation, the step of calculating the initial model gradient of the first GRU model based on the predicted result sharding and the encrypted tag data sharding sent by the central node includes:

[0119] The model loss function is calculated based on the fragmentation of the prediction results and the fragmentation of the encrypted tag data sent by the central node;

[0120] The initial model gradient of the first GRU model is calculated based on the model loss function.

[0121] Specifically, during the joint training of the GRU model, in backpropagation, after each participating node calculates the predicted result fragment, it can receive the encrypted label data fragment sent by the central node, calculate the model loss function based on the predicted result fragment and the encrypted label data fragment, and calculate the initial model gradient of the first GRU model based on the model loss function. This initial model gradient can be understood as the local gradient of the GRU model.

[0122] In practical applications, the mean squared error can be used as the model loss function, and the initial model gradient of the first GRU model can be calculated using the safe matrix multiplication method. The safe matrix multiplication method can be optimized using Beaver triples. The model loss function can be expressed by the following formula.

[0123]

[0124] in, It is a fragment of encrypted tag data sent by the central node.

[0125] In summary, for each participating node, the initial model gradient is calculated by sharding the prediction results and the encrypted label data, which provides a training foundation for the subsequent federated training of the GRU model while ensuring data privacy and security.

[0126] Step 108: Obtain the target noise vector, add the target noise vector to the initial model gradient to obtain the updated model gradient, and send the updated model gradient to the central node so that the central node updates the second GRU model according to the updated model gradient, wherein the second GRU model is deployed on the central node.

[0127] In this model, the second GRU model and the first GRU model can be the same GRU model, deployed on different nodes. The second GRU model deployed on the central node can be trained on the central node. The model parameters of the trained second GRU model can be sent to the participating nodes to update the model parameters of the local first GRU model, thereby achieving federated learning of the GRU models. The target noise vector can be the noise vector whose noise direction is most similar to the gradient direction of the initial model gradient.

[0128] In practical applications, when there are multiple participating nodes, each participating node can obtain the target noise vector, add the target noise vector to the initial model gradient to obtain the updated model gradient, and send the updated model gradient to the central node. The central node aggregates the updated model gradients sent by each participating node and then trains the second GRU model.

[0129] Further, obtaining the target noise vector includes:

[0130] Multiple candidate noise vectors are identified, where each candidate noise vector has a different noise direction;

[0131] Calculate the similarity between the initial model gradient and the plurality of candidate noise vectors, and select the target noise vector from the plurality of candidate noise vectors based on the similarity.

[0132] Specifically, multiple candidate noise vectors with different noise directions can be identified. The similarity between the initial model gradient and each candidate noise vector is calculated, and the candidate noise vector with the highest similarity is determined as the target noise vector. Subsequently, this target noise vector can be used to update the initial model gradient to obtain the updated model gradient.

[0133] In practical applications, a noise direction pool can be created. The noise direction pool can include multiple candidate noise vectors with different noise directions. These candidate noise vectors can be randomly generated using a Gaussian distribution and standardized to unit vectors, such that each candidate noise vector has a unit norm (i.e., standardization). This indicates that each candidate noise vector follows a pattern with a mean of 0 and a variance of . It follows a normal distribution (i.e., a Gaussian distribution).

[0134] The cosine similarity between each candidate noise vector and the gradient of the initial model can be calculated using the following formula.

[0135]

[0136] in, For the initial model gradient, For cosine similarity, For each candidate noise vector.

[0137] The formula for updating the model gradient by using the target noise vector to update the initial model gradient is shown below.

[0138]

[0139] in, To update the model gradient, .

[0140] In summary, by adding a target noise vector to the initial model gradient, differential privacy processing of the initial model gradient is achieved. This target noise vector considers not only the magnitude but also the noise direction. The noise direction of the target noise vector is consistent with the gradient direction of the initial model gradient, thus protecting the gradient while avoiding performance degradation of the GRU model caused by excessive gradient deviation.

[0141] Further, obtaining the target noise vector includes:

[0142] Based on the data sensitivity of the encrypted sample data and the encrypted tag data fragments, the intensity of the target noise vector is adjusted to obtain the adjusted target noise vector.

[0143] Specifically, in each iteration of the first and second GRU models, a dynamic masking mechanism can be used to adjust the strength of the acquired target noise vector (i.e., the mask noise intensity) based on the data sensitivity, thereby obtaining an adjusted target noise vector that balances the accuracy and security of the GRU model.

[0144] In practical applications, a dynamic masking mechanism can also be used to encrypt and aggregate the hidden state at the current time step, where the masking noise intensity is adaptively adjusted according to the feature sensitivity.

[0145] Furthermore, after sending the updated model gradient to the central node, the process further includes:

[0146] The system receives updated model parameters of the second GRU model sent by the central node, wherein the updated model parameters are obtained by the central node after updating the second GRU model according to the updated model gradient.

[0147] Based on the updated model parameters of the second GRU model, the model parameters of the first GRU model are adjusted to obtain the adjusted first GRU model.

[0148] Continue executing the steps of determining encrypted sample data, splitting the encrypted sample data to obtain multiple encrypted sample data fragments, until a second GRU model that meets the training stopping condition is obtained.

[0149] Specifically, the central node can aggregate the updated model gradients sent by each participating node according to the multi-party security protocol (i.e., MPC protocol parameters) to obtain the global gradient. It then uses this global gradient to adjust the model parameters of the second GRU model, obtaining the updated model parameters, and sends these updated parameters to each participating node. Each participating node can then adjust the model parameters of its locally deployed first GRU model based on the updated parameters of the second GRU model, obtaining the adjusted first GRU model. This allows for iterative training of the GRU model in the next round.

[0150] In practical applications, the updated model gradients calculated by each participating node are encrypted and transmitted to the central node via secret sharing, or gradients are aggregated jointly by multiple participants. The MPC protocol ensures that each participating node can only see the model gradients it has calculated and cannot access the private data of other parties. Participants aggregate the encrypted model gradients through a secure computation protocol to obtain the global gradient.

[0151] Training stopping conditions can be met if the number of training iterations reaches a preset threshold and / or the model loss function converges. For details, see [link to documentation]. Figure 2 , Figure 2 The diagram illustrates a process for training a vertical federated GRU model based on secret sharing and differential privacy, according to an embodiment of this specification, applied to participating nodes and a central node, specifically including the following steps.

[0152] Step 202: System initialization, the central node and each participating node negotiate MPC parameters, initialize the model weights of the GRU model, standardize the sample data and label data, and initialize the noise pool.

[0153] Step 204: For each participating node, calculate the update gate, reset gate, candidate hidden state, and hidden state of the GRU model at the current time step.

[0154] Step 206: For each participating node, obtain the prediction result fragments output by the GRU model.

[0155] Step 208: The central node sends encrypted tag data fragments to each participating node.

[0156] Step 210: Each participating node calculates the model loss function based on the prediction results and the encrypted label data.

[0157] Step 212: Determine whether the model loss function has converged. If yes, proceed to step 214; otherwise, proceed to step 216.

[0158] Step 214: End training.

[0159] Step 216: For each participating node, use the optimizer to calculate the initial model gradient of the GRU model.

[0160] Step 218: Perform differential privacy processing on the initial model gradient after optimization to obtain the updated model gradient.

[0161] Step 220: Aggregate the gradients of the updated model calculated by each participating node to obtain the global gradient, and adjust the model parameters of the GRU model according to the global gradient.

[0162] In summary, the vertical federated GRU model training method based on secret sharing and differential privacy provided in this specification optimizes computational efficiency through a secret sharing protocol and improves the compatibility of secret sharing with GRU while ensuring privacy. A segmented secret sharing protocol is designed in the GRU reset / update gate calculation (i.e., the activation function is split into a polynomial approximation + secret sharing), instead of directly applying the general SS protocol, and Beaver triples are used to reduce the computational cost of model training. Regarding the secure aggregation of hidden states, a dynamic masking mechanism is introduced. In each iteration, the noise intensity of the target noise vector for secret sharing is adaptively adjusted according to data sensitivity, balancing accuracy and security, and considering not only the noise amplitude but also the noise direction.

[0163] In summary, the above method, when performing federated training on the GRU model, can split the encrypted sample data into multiple encrypted sample data slices, making it impossible for any party to reconstruct the original encrypted sample data from a single encrypted sample data slice. These multiple encrypted sample data slices are input into the first GRU model to obtain prediction result slices corresponding to each slice. Based on these prediction result slices and the encrypted label data slices sent by the central node, the initial model gradient of the first GRU model is calculated, and the target noise vector is obtained. This target noise vector is added to the initial model gradient to obtain the updated model gradient, thus achieving secondary encryption of the model gradient. This secondary encrypted updated model gradient is then sent to the central node, enabling the central node to update the second GRU model based on the updated model gradient, thereby facilitating subsequent federated training of the second GRU model. By bringing the sample data encryption process forward, the parameter calculation process in the first GRU model is directly performed based on the encrypted sample data slices, avoiding the large amount of computational resources consumed by encrypting intermediate results. This improves the federated learning efficiency of the GRU model and ensures the privacy and security of the training data through secondary encryption.

[0164] Corresponding to the above method embodiments, this specification also provides a method for training a vertical federated GRU model based on secret sharing and differential privacy, applied to the central node. See [link to documentation]. Figure 3 , Figure 3 A flowchart is shown of another method for training a longitudinal federated GRU model based on secret sharing and differential privacy according to an embodiment of this specification, specifically including the following steps.

[0165] Step 302: Determine the encrypted tag data, split the encrypted tag data, and obtain multiple encrypted tag data fragments;

[0166] Step 304: Send the multiple encrypted tag data fragments to the participating nodes, and receive the updated model gradient of the first GRU model sent by the participating nodes, wherein the first GRU model is deployed on the participating nodes, and the updated model gradient is obtained by adding a target noise vector to the initial model gradient, and the initial model gradient is calculated based on the multiple encrypted tag data fragments;

[0167] Step 306: Update the second GRU model according to the updated model gradient to obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed at the central node.

[0168] In one optional embodiment, there are multiple participating nodes;

[0169] The step of updating the second GRU model according to the updated model gradient to obtain the updated second GRU model includes:

[0170] The gradients of the updated models sent by multiple participating nodes are aggregated to obtain the gradient of the target model.

[0171] The second GRU model is updated based on the gradient of the target model to obtain the updated second GRU model.

[0172] The target model gradient can be understood as the global gradient. The central node and participating nodes can both be understood as the central node and participating nodes in the above-described longitudinal federated GRU model training method based on secret sharing and differential privacy; these will not be repeated in the embodiments of this specification.

[0173] The above is an illustrative scheme of a vertical federated GRU model training method based on secret sharing and differential privacy according to this embodiment. It should be noted that the technical solution of this vertical federated GRU model training method based on secret sharing and differential privacy belongs to the same concept as the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above. Details not described in detail in the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy can be found in the description of the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above.

[0174] The following is in conjunction with the appendix Figure 4 Taking the application of the vertical federated GRU model training method based on secret sharing and differential privacy provided in this specification in the central node and participating nodes as an example, the training method of the vertical federated GRU model based on secret sharing and differential privacy will be further explained. Figure 4 The present specification illustrates a flowchart of a training method for a longitudinal federated GRU model based on secret sharing and differential privacy, according to an embodiment of this specification, which specifically includes the following steps.

[0175] Step 402: The central node determines the encrypted tag data, splits the encrypted tag data, and obtains multiple encrypted tag data fragments.

[0176] Step 404: The central node sends multiple encrypted tag data fragments to the participating nodes.

[0177] Step 406: Participating nodes determine the encrypted sample data, split the encrypted sample data to obtain multiple encrypted sample data fragments; input the multiple encrypted sample data fragments into the first GRU model to obtain the prediction result fragments corresponding to the multiple encrypted sample data fragments; calculate the initial model gradient of the first GRU model based on the prediction result fragments and the encrypted label data fragments sent by the central node; obtain the target noise vector, add the target noise vector to the initial model gradient, and obtain the updated model gradient.

[0178] Step 408: Participating nodes send updated model gradients to the central node.

[0179] Step 410: At the central node, update the second GRU model according to the updated model gradient to obtain the updated second GRU model.

[0180] Step 412: The central node sends the updated model parameters of the updated second GRU model to the participating nodes.

[0181] Step 414: Participating nodes update the model parameters of the first GRU model according to the updated model parameters to obtain the updated first GRU model.

[0182] Understandably, steps 402 to 414 above can be understood as a round of federated training for the first GRU model and the second GRU model, and the above steps can be repeated for the next round of federated training.

[0183] In the above method, when performing federated training on the GRU model, the encrypted sample data can be split into multiple encrypted sample data slices, making it impossible for any party to reconstruct the original encrypted sample data from a single encrypted sample data slice. These multiple encrypted sample data slices are input into the first GRU model to obtain prediction result slices corresponding to each slice. Based on these prediction result slices and the encrypted label data slices sent by the central node, the initial model gradient of the first GRU model is calculated, and the target noise vector is obtained. This target noise vector is added to the initial model gradient to obtain the updated model gradient, thus achieving secondary encryption of the model gradient. This secondary encrypted updated model gradient is then sent to the central node, allowing the central node to update the second GRU model based on the updated model gradient, enabling subsequent federated training of the second GRU model. By bringing the sample data encryption process forward, the parameter calculation process in the first GRU model is directly performed based on the encrypted sample data slices, avoiding the large amount of computational resources consumed by encrypting intermediate results. This improves the federated learning efficiency of the GRU model and ensures the privacy and security of the training data through secondary encryption.

[0184] Corresponding to the above method embodiments, this specification also provides an embodiment of a training device for a vertical federated GRU model based on secret sharing and differential privacy, applied to participating nodes. Figure 5 This diagram illustrates a structural schematic of a vertical federated GRU model training device based on secret sharing and differential privacy, according to one embodiment of this specification. Figure 5 As shown, the device includes:

[0185] The splitting module 502 is configured to determine the encrypted sample data, split the encrypted sample data, and obtain multiple encrypted sample data fragments.

[0186] Input module 504 is configured to input the plurality of encrypted sample data shards into a first GRU model to obtain prediction result shards corresponding to the plurality of encrypted sample data shards, wherein the first GRU model is deployed on the participating node;

[0187] The calculation module 506 is configured to calculate the initial model gradient of the first GRU model based on the fragmentation of the prediction result and the fragmentation of the encrypted tag data sent by the central node.

[0188] The sending module 508 is configured to acquire a target noise vector, add the target noise vector to the initial model gradient to obtain an updated model gradient, and send the updated model gradient to the central node so that the central node updates the second GRU model according to the updated model gradient, wherein the second GRU model is deployed on the central node.

[0189] In an optional embodiment, the sending module 508 is further configured to:

[0190] Multiple candidate noise vectors are identified, where each candidate noise vector has a different noise direction;

[0191] Calculate the similarity between the initial model gradient and the plurality of candidate noise vectors, and select the target noise vector from the plurality of candidate noise vectors based on the similarity.

[0192] In an optional embodiment, the input module 504 is further configured to:

[0193] The multiple encrypted sample data are fragmented and input into the first GRU model. Based on the multiple encrypted sample data, the polynomial corresponding to the first activation function, and the hidden state of the previous time step, the update gate and reset gate of the first GRU model are calculated.

[0194] Based on the reset gate, the polynomial corresponding to the second activation function, and the hidden state of the previous time step, calculate the candidate hidden state corresponding to the first GRU model;

[0195] Calculate the hidden state at the current time step based on the update gate and the candidate hidden states;

[0196] Based on the hidden state of the current time step, calculate and output the prediction result fragments corresponding to the multiple encrypted sample data fragments.

[0197] In an optional embodiment, the input module 504 is further configured to:

[0198] The first activation function is approximated as the polynomial corresponding to the first activation function;

[0199] The second activation function can be approximated as the polynomial corresponding to the second activation function.

[0200] In an optional embodiment, the computing module 506 is further configured to:

[0201] The model loss function is calculated based on the fragmentation of the prediction results and the fragmentation of the encrypted tag data sent by the central node;

[0202] The initial model gradient of the first GRU model is calculated based on the model loss function.

[0203] In an optional embodiment, the splitting module 502 is further configured to:

[0204] Obtain the sample data stored in the participating nodes;

[0205] The sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

[0206] In an optional embodiment, the splitting module 502 is further configured to:

[0207] The sample data is standardized to obtain standardized sample data;

[0208] The step of encrypting the sample data according to a preset encryption algorithm to obtain encrypted sample data includes:

[0209] The standardized sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

[0210] In an optional embodiment, the sending module 508 is further configured to:

[0211] The system receives updated model parameters of the second GRU model sent by the central node, wherein the updated model parameters are obtained by the central node after updating the second GRU model according to the updated model gradient.

[0212] Based on the updated model parameters of the second GRU model, the model parameters of the first GRU model are adjusted to obtain the adjusted first GRU model.

[0213] Continue executing the steps of determining encrypted sample data, splitting the encrypted sample data to obtain multiple encrypted sample data fragments, until a second GRU model that meets the training stopping condition is obtained.

[0214] In an optional embodiment, the sending module 508 is further configured to:

[0215] Based on the data sensitivity of the encrypted sample data and the encrypted tag data fragments, the intensity of the target noise vector is adjusted to obtain the adjusted target noise vector.

[0216] In the aforementioned device, during federated training of the GRU model, the encrypted sample data can be split into multiple encrypted sample data slices. This prevents any single party from reconstructing the original encrypted sample data from a single encrypted sample data slice. These multiple encrypted sample data slices are input into the first GRU model to obtain prediction result slices corresponding to each slice. Based on these prediction result slices and the encrypted label data slices sent by the central node, the initial model gradient of the first GRU model is calculated. The target noise vector is then obtained and added to the initial model gradient to obtain an updated model gradient, thus achieving secondary encryption of the model gradient. This secondary-encrypted updated model gradient is then sent to the central node, enabling the central node to update the second GRU model based on the updated model gradient. This facilitates subsequent federated training of the second GRU model. By bringing the sample data encryption process forward, the parameter calculation process in the first GRU model is directly performed based on the encrypted sample data slices, avoiding the large amount of computational resources consumed by encrypting intermediate results. This improves the federated learning efficiency of the GRU model and ensures the privacy and security of the training data through secondary encryption.

[0217] The above is an illustrative scheme of a vertical federated GRU model training device based on secret sharing and differential privacy according to this embodiment. It should be noted that the technical solution of this vertical federated GRU model training device based on secret sharing and differential privacy belongs to the same concept as the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above. Details not described in detail in the technical solution of the vertical federated GRU model training device based on secret sharing and differential privacy can be found in the description of the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above.

[0218] Corresponding to the above method embodiments, this specification also provides an embodiment of a training device for a vertical federated GRU model based on secret sharing and differential privacy, applied to the central node. Figure 6 This specification illustrates a schematic diagram of another vertical federated GRU model training device based on secret sharing and differential privacy, provided in one embodiment. Figure 6 As shown, the device includes:

[0219] The splitting module 602 is configured to determine the encrypted tag data, split the encrypted tag data, and obtain multiple encrypted tag data fragments;

[0220] The communication module 604 is configured to send the multiple encrypted tag data fragments to the participating nodes and receive the updated model gradient of the first GRU model sent by the participating nodes, wherein the first GRU model is deployed on the participating nodes, and the updated model gradient is obtained by adding a target noise vector to the initial model gradient, and the initial model gradient is calculated based on the multiple encrypted tag data fragments.

[0221] The update module 606 is configured to update the second GRU model according to the update model gradient, obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed at the central node.

[0222] In one optional embodiment, there are multiple participating nodes;

[0223] The update module 606 is further configured as follows:

[0224] The gradients of the updated models sent by multiple participating nodes are aggregated to obtain the gradient of the target model.

[0225] The second GRU model is updated based on the gradient of the target model to obtain the updated second GRU model.

[0226] The above is an illustrative scheme of a vertical federated GRU model training device based on secret sharing and differential privacy according to this embodiment. It should be noted that the technical solution of this vertical federated GRU model training device based on secret sharing and differential privacy belongs to the same concept as the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above. Details not described in detail in the technical solution of the vertical federated GRU model training device based on secret sharing and differential privacy can be found in the description of the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above.

[0227] Corresponding to the above method embodiments, this specification also provides a training system for a vertical federated GRU model based on secret sharing and differential privacy, including participating nodes and a central node, wherein,

[0228] The central node is configured to determine encrypted tag data, split the encrypted tag data to obtain multiple encrypted tag data fragments, and send the multiple encrypted tag data fragments to the participating nodes;

[0229] The participating node is configured to: determine encrypted sample data; split the encrypted sample data to obtain multiple encrypted sample data fragments; input the multiple encrypted sample data fragments into a first GRU model to obtain prediction result fragments corresponding to the multiple encrypted sample data fragments; calculate the initial model gradient of the first GRU model based on the prediction result fragments and the encrypted label data fragments sent by the central node; obtain a target noise vector; add the target noise vector to the initial model gradient to obtain an updated model gradient; and send the updated model gradient to the central node, wherein the first GRU model is deployed on the participating node.

[0230] The central node is configured to update the second GRU model according to the updated model gradient, obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed on the central node.

[0231] The above is an illustrative scheme of a vertical federated GRU model training system based on secret sharing and differential privacy according to this embodiment. It should be noted that the technical solution of this vertical federated GRU model training system based on secret sharing and differential privacy belongs to the same concept as the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above. Details not described in detail in the technical solution of the vertical federated GRU model training system based on secret sharing and differential privacy can be found in the description of the technical solution of the vertical federated GRU model training method based on secret sharing and differential privacy described above.

[0232] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0233] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0234] In one embodiment of this application, the aforementioned components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0235] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0236] The processor 720 is used to execute the following computer program / instructions, which, when executed by the processor, implement the steps of the above method.

[0237] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computing device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0238] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0239] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer-readable storage medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0240] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0241] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above method.

[0242] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0243] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0244] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0245] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0246] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A training method for a vertical federated GRU model based on secret sharing and differential privacy, applied to participating nodes, including: The encrypted sample data is determined, and the encrypted sample data is split to obtain multiple encrypted sample data fragments, wherein the encrypted sample data is the encrypted sample data held and provided by the participating nodes; The multiple encrypted sample data shards are input into the first GRU model to obtain the prediction result shards corresponding to the multiple encrypted sample data shards. The first GRU model is deployed on the participating node. The forward propagation process of the first GRU model is completed on the multiple encrypted sample data shards. The prediction result shards are calculated by the first GRU model on the multiple encrypted sample data shards according to the secret sharing protocol. Based on the predicted result fragments and the encrypted tag data fragments sent by the central node, calculate the initial model gradient of the first GRU model; A target noise vector is obtained, and the target noise vector is added to the initial model gradient to obtain an updated model gradient. The updated model gradient is then sent to the central node so that the central node updates the second GRU model according to the updated model gradient. The second GRU model is deployed on the central node.

2. The method according to claim 1, wherein obtaining the target noise vector comprises: Multiple candidate noise vectors are identified, where each candidate noise vector has a different noise direction; Calculate the similarity between the initial model gradient and the plurality of candidate noise vectors, and select the target noise vector from the plurality of candidate noise vectors based on the similarity.

3. The method according to claim 1, wherein inputting the plurality of encrypted sample data slices into the first GRU model to obtain prediction result slices corresponding to the plurality of encrypted sample data slices comprises: The multiple encrypted sample data fragments are input into the first GRU model. Based on the multiple encrypted sample data fragments, the polynomial corresponding to the first activation function, and the hidden state of the previous time step, the update gate and reset gate of the first GRU model are calculated. Based on the reset gate, the polynomial corresponding to the second activation function, and the hidden state of the previous time step, calculate the candidate hidden state corresponding to the first GRU model; Calculate the hidden state at the current time step based on the update gate and the candidate hidden states; Based on the hidden state of the current time step, calculate and output the prediction result fragments corresponding to the multiple encrypted sample data fragments.

4. The method according to claim 3, further comprising, before calculating the update gate and reset gate of the first GRU model based on the plurality of encrypted sample data fragments, the polynomial corresponding to the first activation function, and the hidden state of the previous time step: The first activation function is approximated as the polynomial corresponding to the first activation function; The second activation function can be approximated as the polynomial corresponding to the second activation function.

5. The method according to claim 1, wherein calculating the initial model gradient of the first GRU model based on the predicted result sharding and the encrypted tag data sharding sent by the central node includes: The model loss function is calculated based on the fragmentation of the prediction results and the fragmentation of the encrypted tag data sent by the central node; The initial model gradient of the first GRU model is calculated based on the model loss function.

6. The method according to any one of claims 1-5, wherein determining the encrypted sample data comprises: Obtain the sample data stored in the participating nodes; The sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

7. The method according to claim 6, further comprising, after obtaining the sample data stored in the participating nodes: The sample data is standardized to obtain standardized sample data; The step of encrypting the sample data according to a preset encryption algorithm to obtain encrypted sample data includes: The standardized sample data is encrypted according to a preset encryption algorithm to obtain encrypted sample data.

8. The method according to any one of claims 1-5, wherein after sending the updated model gradient to the central node, it further comprises: The system receives updated model parameters of the second GRU model sent by the central node, wherein the updated model parameters are obtained by the central node after updating the second GRU model according to the updated model gradient. Based on the updated model parameters of the second GRU model, the model parameters of the first GRU model are adjusted to obtain the adjusted first GRU model. Continue executing the steps of determining encrypted sample data, splitting the encrypted sample data to obtain multiple encrypted sample data fragments, until a second GRU model that meets the training stopping condition is obtained.

9. The method according to any one of claims 1-5, wherein obtaining the target noise vector comprises: Based on the data sensitivity of the encrypted sample data and the encrypted tag data fragments, the intensity of the target noise vector is adjusted to obtain the adjusted target noise vector.

10. A training method for a vertical federated GRU model based on secret sharing and differential privacy, applied to the central node, including: The encrypted tag data is determined, and the encrypted tag data is split to obtain multiple encrypted tag data fragments, wherein the encrypted tag data is the encrypted tag data held and provided by the central node; The multiple encrypted label data shards are sent to the participating nodes, and the updated model gradient of the first GRU model sent by the participating nodes is received. The first GRU model is deployed on the participating nodes. The updated model gradient is obtained by adding a target noise vector to the initial model gradient. The initial model gradient is calculated based on the multiple encrypted label data shards and the prediction result shards. The forward propagation process of the first GRU model is completed on the multiple encrypted sample data shards. The prediction result shards are calculated by the first GRU model on the multiple encrypted sample data shards according to the secret sharing protocol. The second GRU model is updated according to the updated model gradient to obtain the updated second GRU model, and the updated model parameters of the updated second GRU model are sent to the participating nodes, wherein the second GRU model is deployed at the central node.

11. The method according to claim 10, wherein the participating nodes are multiple; The step of updating the second GRU model according to the updated model gradient to obtain the updated second GRU model includes: The gradients of the updated models sent by multiple participating nodes are aggregated to obtain the gradient of the target model. The second GRU model is updated based on the gradient of the target model to obtain the updated second GRU model.

12. A training system for a vertical federated GRU model based on secret sharing and differential privacy, comprising participating nodes and a central node, wherein, The central node is configured to determine encrypted tag data, split the encrypted tag data to obtain multiple encrypted tag data fragments, and send the multiple encrypted tag data fragments to the participating nodes; The participating node is configured to determine encrypted sample data, split the encrypted sample data to obtain multiple encrypted sample data slices, and input the multiple encrypted sample data slices into a first GRU model to obtain prediction result slices corresponding to the multiple encrypted sample data slices. Based on the predicted result fragments and the encrypted tag data fragments sent by the central node, calculate the initial model gradient of the first GRU model; A target noise vector is obtained, and the target noise vector is added to the initial model gradient to obtain an updated model gradient. The updated model gradient is then sent to the central node. The first GRU model is deployed on the participating nodes, and the encrypted sample data is encrypted sample data held and provided by the participating nodes. The forward propagation process of the first GRU model is completed on the multiple encrypted sample data shards, and the prediction result shard is calculated by the first GRU model on the multiple encrypted sample data shards according to the secret sharing protocol. The central node is configured to update the second GRU model according to the updated model gradient, obtain the updated second GRU model, and send the updated model parameters of the updated second GRU model to the participating nodes, wherein the second GRU model is deployed on the central node.

13. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 11.

15. A computer program product comprising computer programs / instructions that, when executed by a processor, implement the steps of the method of any one of claims 1 to 11.

Images