Privacy protection based decentralized longitudinal federated learning model training method

Abstract

Embodiments of the present specification provide a privacy protection based decentralized longitudinal federated learning model training method. The method can be executed by a plurality of first nodes, comprising: obtaining intermediate results respectively provided by a plurality of second nodes, which are obtained by performing forward propagation training on a local model corresponding to a target model by the second nodes; determining a score corresponding to each intermediate result for representing reliability based on label data of the plurality of samples, selecting a plurality of first intermediate results whose scores reach a threshold value from each intermediate result; determining a first prediction loss value of the target model based on an aggregation result of the plurality of first intermediate results and the label data; in response to the first prediction loss value not satisfying a model convergence condition, determining a gradient value corresponding to each of a plurality of third nodes that provide the plurality of first intermediate results based on the first prediction loss value and a current first model parameter of a local model of the third node; and sending the gradient value to the plurality of third nodes.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of computer technology, and in particular relate to a method for training a privacy-preserving decentralized longitudinal federated learning model. Background Technology

[0002] With the rapid development of information technology, most enterprises possess a large amount of user data. Sharing user data among enterprises would better support their assessment of user status and needs, thereby enabling them to provide more personalized services. However, due to concerns about data security and privacy breaches, some enterprises are hesitant to directly disclose or share their user data. Summary of the Invention

[0003] The purpose of this invention is to provide a privacy-preserving decentralized longitudinal federated learning model training scheme that enables multiple participants with the same sample space but different feature spaces to participate in the same model training task, while ensuring data security and preventing privacy leaks as much as possible, and ensuring the reliability of the target model being trained.

[0004] This specification provides a privacy-preserving decentralized longitudinal federated learning model training method, executed by multiple first nodes, each storing label data corresponding to multiple samples of a target model. The method includes: acquiring intermediate results provided by multiple second nodes, wherein the intermediate results are obtained by the second nodes performing forward propagation training on a local model corresponding to the target model based on the feature data of the multiple samples stored locally, and the multiple second nodes storing feature data of different dimensions of the multiple samples; determining a score for representing reliability for each intermediate result based on the label data, and selecting multiple first intermediate results whose scores reach a threshold from the intermediate results; determining a first prediction loss value for the target model based on the aggregation result of the multiple first intermediate results and the label data; in response to the first prediction loss value not satisfying the model convergence condition, determining gradient values ​​corresponding to the multiple third nodes based on the first prediction loss value and the current first model parameters of the local models of the multiple third nodes that provided the multiple first intermediate results; and sending the corresponding gradient values ​​to the multiple third nodes respectively.

[0005] A second aspect of this specification provides a privacy-preserving decentralized longitudinal federated learning model training device, applied to one of a plurality of first nodes, wherein the plurality of first nodes store label data corresponding to a plurality of samples of a target model. The device includes: an acquisition unit configured to acquire intermediate results provided by a plurality of second nodes, wherein the intermediate results are obtained by the second nodes performing forward propagation training on a local model corresponding to the target model based on feature data of the plurality of samples stored locally, wherein the plurality of second nodes store feature data of different dimensions of the plurality of samples; and a selection unit configured to determine, based on the label data, a selection unit for each intermediate result corresponding to a table. The system assigns a reliability score to the target model and selects multiple first intermediate results whose scores reach a threshold from the various intermediate results. A loss determination unit is configured to determine a first predicted loss value for the target model based on the aggregation result of the multiple first intermediate results and the label data. A gradient determination unit is configured to, in response to the first predicted loss value not satisfying the model convergence condition, determine gradient values ​​corresponding to the multiple third nodes based on the first predicted loss value and the current first model parameters of the local models of the multiple third nodes that provide the multiple first intermediate results. A sending unit is configured to send the corresponding gradient value to the third node among the multiple third nodes.

[0006] A third aspect of this specification provides a computer-readable storage medium having a computer program stored thereon that, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0007] A fourth aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in the first aspect.

[0008] The fifth aspect of this specification provides a computer program product that, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0009] In the scheme provided in the embodiments of this specification, multiple second nodes have the same sample space but different feature spaces, that is, the multiple second nodes store feature data of multiple samples (from the same batch of samples) in different dimensions. During the model training process of the multiple second nodes, the multiple second nodes can perform forward propagation training on the local model corresponding to the target model based on the feature data of the multiple samples stored locally to obtain intermediate results, and provide the intermediate results to the multiple first nodes. This ensures that the feature data does not leave the domain, thereby ensuring data security and preventing privacy leakage. Next, the multiple first nodes can determine the reliability score corresponding to each intermediate result based on the label data of the multiple samples corresponding to the target model, and select multiple first intermediate results whose scores reach a threshold. Based on the aggregation result of these multiple first intermediate results and the label data, a first predicted loss value for the target model is determined. Then, in response to the first predicted loss value not satisfying the model convergence condition, the gradient values ​​corresponding to each of the multiple third nodes that provided the multiple first intermediate results are determined based on the first predicted loss value and the current first model parameters of their respective local models. The corresponding gradient values ​​are then sent to each of the multiple third nodes, allowing them to update their local model parameters based on the obtained gradient values. This allows intermediate results whose scores do not reach the threshold to be considered unreliable (invalid), preventing them from participating in the final aggregation process and preventing the second node providing the intermediate results from performing subsequent model training, thus providing good reliability for the target model. Furthermore, model aggregation based on multiple first nodes rather than a single server provides strong resistance to single-point attacks and single-point failures, effectively preventing the leakage of intermediate results and training process data. Therefore, this scheme enables multiple participants with the same sample space but different feature spaces to perform the same model training task, while ensuring data security and preventing privacy leaks as much as possible, and ensuring the reliability of the target model being trained. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a diagram of a blockchain architecture in one embodiment;

[0012] Figure 2 This is a schematic diagram illustrating one application scenario in which the embodiments of this specification can be applied;

[0013] Figure 3 This is a schematic diagram of the initialization process of the blockchain system in the embodiments of this specification;

[0014] Figure 4 This is a schematic diagram of the model training initialization process in the embodiments of this specification;

[0015] Figure 5 This is a timeline diagram of a privacy-preserving decentralized longitudinal federated learning model training method in the embodiments of this specification;

[0016] Figure 6 This is a schematic diagram of the decentralized vertical federated learning model training method based on consortium blockchain in the embodiments of this specification;

[0017] Figure 7 This is a schematic diagram of the decentralized vertical federated learning model training method based on consortium blockchain in the embodiments of this specification;

[0018] Figure 8 This is a schematic diagram of the structure of the privacy-preserving decentralized longitudinal federated learning model training device in the embodiments of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0020] As mentioned earlier, with the rapid development of information technology, most enterprises possess a large amount of user data. Sharing user data among enterprises can better support their assessment of user status and needs, thereby providing more personalized services. However, due to concerns about data security and privacy leaks, some enterprises cannot directly disclose or share their user data. Traditional centralized machine learning methods have certain data security and privacy leakage issues.

[0021] As a form of distributed machine learning, federated learning offers a new approach to collaboratively training models by parties with data remaining within their domain. In federated learning, parties collaborate on training by sharing the model rather than the data itself, thus protecting data security to some extent. Federated learning is broadly categorized into horizontal federated learning, vertical federated learning, and transfer learning. Vertical federated learning typically refers to participants with the same sample space but different feature spaces collaboratively training a model while ensuring data remains within its domain. Specifically, participants in vertical federated learning can possess feature data from different dimensions of the same user group; for example, one party might hold credit records for the same user group, while another holds their consumption records. Vertical federated learning primarily expands feature data through sample alignment, thereby improving model performance. In real-world applications, vertical federated learning has received widespread attention.

[0022] Traditional vertical federated learning methods are based on centralized devices, where each participant communicates with a single server responsible for distributing federated learning tasks and generating the final model training results. However, in collaborative environments involving multiple enterprises and institutions, continuing with this centralized approach makes the server's single point of failure highly susceptible to data model leakage, potentially leading to the disclosure of confidential corporate data. Furthermore, there is the risk of participants poisoning the training results or even directly interfering with the server's aggregation process. Therefore, traditional vertical federated learning methods have several shortcomings, necessitating changes to their architecture and execution processes to ensure the security and stability of this learning approach.

[0023] This specification provides a privacy-preserving decentralized longitudinal federated learning model training method, involving multiple first nodes and multiple second nodes. The multiple first nodes store label data corresponding to multiple samples of the target model, and the multiple second nodes store local models corresponding to the target model and feature data of different dimensions of the multiple samples. The multiple second nodes can perform forward propagation training on their local models based on the locally stored feature data of the multiple samples to obtain intermediate results, and provide these intermediate results to the multiple first nodes. Next, the multiple first nodes can determine a reliability score corresponding to each intermediate result based on the label data, and select multiple first intermediate results whose scores reach a threshold. Based on the aggregation result of these multiple first intermediate results and the label data, a first prediction loss value for the target model is determined. Then, in response to the first prediction loss value not satisfying the model convergence condition, based on the first prediction loss value and the current first model parameters of the local models of the multiple third nodes that provided the multiple first intermediate results, gradient values ​​corresponding to each of the multiple third nodes are determined, and their corresponding gradient values ​​are sent to the multiple third nodes. Finally, the multiple third nodes can update the parameters of their local models based on the obtained gradient values.

[0024] The model training method provided in the above embodiments of this specification enables multiple participants with the same sample space but different feature spaces to perform the same model training task, while ensuring data security and preventing privacy leaks as much as possible, and ensuring the reliability of the target model being trained.

[0025] In one implementation, the aforementioned multiple first nodes can be multiple centralized nodes, such as multiple centralized servers. Model aggregation is achieved based on multiple centralized servers rather than a single server, which has strong resistance to single point attacks and single point failures and can effectively prevent the leakage of intermediate results and training process data.

[0026] It should be noted that, in the case where the aforementioned multiple first nodes are multiple centralized nodes, in the solution provided in the embodiments of this specification, the device of the model user can publish a model training task and an initial model to the aforementioned multiple first nodes. After the aforementioned multiple first nodes determine that they accept the model training task, they can use the obtained initial model as the target model and send the initial model to the multiple second nodes participating in the execution of the task, so that the multiple second nodes can use the initial model as their local model for model training. Specific implementation details can be found in the relevant descriptions below, for example... Figure 4 , Figure 5 The relevant descriptions in the corresponding embodiments.

[0027] In practice, blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. In a blockchain, data blocks are sequentially linked together in a chain-like data structure, and cryptographic methods are used to ensure the immutability and forgery resistance of these data blocks. Due to its decentralized, immutable, and autonomous characteristics, blockchain is receiving increasing attention and application.

[0028] Figure 1 A blockchain architecture diagram from one embodiment is shown. Figure 1 In the blockchain architecture diagram shown, blockchain 100 includes N nodes. Figure 1 The diagram illustrates nodes 1 through 8. The lines connecting the nodes schematically represent P2P (Peer-to-Peer) connections, such as TCP connections, used for data transfer between nodes. These nodes can store the entire ledger, i.e., the state of all blocks and all accounts. Each node in the blockchain can produce the same state by executing the same transactions, and each node can store the same state database.

[0029] In the blockchain field, a transaction refers to a unit of task executed and recorded within the blockchain. A transaction typically includes a From field, a To field, and a Data field. Specifically, in the case of a transfer transaction, the From field represents the account address initiating the transaction (i.e., initiating a transfer task to another account), the To field represents the account address receiving the transaction (i.e., receiving the transfer), and the Data field includes the transfer amount.

[0030] Blockchain provides the functionality of smart contracts. A smart contract on the blockchain is a contract that can be triggered and executed through transactions within the blockchain system. Smart contracts can be defined in the form of code. Calling a smart contract on the blockchain involves initiating a transaction pointing to the smart contract's address, causing each node in the blockchain to run the smart contract code in a distributed manner.

[0031] In a contract deployment scenario, for example, Bob sends a transaction containing information about creating a smart contract (i.e., deploying the contract) to a server such as... Figure 1 In the blockchain shown, the `data` field of the transaction includes the code (such as bytecode or machine code) of the contract to be created, and the `to` field of the transaction is empty, indicating that the transaction is used to deploy the contract. After the nodes reach an agreement through the consensus mechanism, they determine the contract address "0x6f8ae93…". Each node adds a contract account corresponding to the contract address of the smart contract to the state database, allocates state storage corresponding to the contract account, stores the contract code, and saves the hash value of the contract code in the contract's state storage, thus the contract is successfully created.

[0032] In scenarios where contracts are invoked, for example, Bob sends a transaction to invoke a smart contract, such as... Figure 1 In the blockchain shown, the `from` field of this transaction is the address of the account of the transaction initiator (i.e., Bob), the `to` field is the aforementioned "0x6f8ae93…", which is the address of the smart contract being invoked, and the `data` field of the transaction includes the method and parameters for invoking the smart contract. After consensus is reached on this transaction in the blockchain, each node in the blockchain can execute the transaction, thereby executing the contract separately, and updating the state database based on the execution of the contract.

[0033] A consortium blockchain refers to a decentralized architecture with an access control mechanism, primarily composed of a limited number of participants. Participants in a consortium blockchain must be authenticated before they can verify each other's data and share it. Apart from authorized nodes, other nodes cannot join the consortium blockchain. Consortium blockchains are widely used in inter-enterprise data sharing and collaboration, offering greater security, stability, and processing power compared to public blockchains.

[0034] As a decentralized architecture, consortium blockchains offer inventors a new approach to solving the single point of failure problem inherent in traditional vertical federated learning.

[0035] Specifically, in one implementation, the aforementioned plurality of first nodes and plurality of second nodes can be included in a blockchain system (e.g., a consortium blockchain system). Thus, the embodiments of this specification can provide a model training scheme based on a blockchain system. This scheme, based on a decentralized blockchain system rather than a single server, has strong resistance to single-point attacks and single-point failures, and can effectively prevent the leakage of intermediate results and training process data. Furthermore, the blockchain system can provide an effective access control mechanism, excluding malicious nodes from the blockchain system and preventing malicious nodes from interfering with the federated learning process or even stealing training data.

[0036] Below, in conjunction with Figure 2 This document describes an embodiment of the present specification that can be applied to one application scenario. Figure 2 This is a schematic diagram illustrating one application scenario in which the embodiments of this specification can be applied.

[0037] exist Figure 2 The application scenario shown may include device 201 of the model user, and nodes of a finite number of participating parties. Figure 2 The blockchain system 202, illustrated in the diagram, consists of nodes 1-8, forming a blockchain system. Blockchain system 202 includes multiple first nodes ( Figure 2 The diagram illustrates nodes 1, 2, and 8 as the first nodes and multiple second nodes. Figure 2 The diagram illustrates nodes 3-5 as second nodes. These first nodes store label data for multiple samples corresponding to the target model, while the second nodes store the local model corresponding to the target model and feature data of different dimensions for the multiple samples. The initial target model is the initial model published by device 201 to blockchain system 202. The label data is provided by device 201 to the first nodes, and the initial local model of each second node is the initial model sent by the first nodes.

[0038] In one implementation, the plurality of first nodes and the plurality of second nodes can be determined from the nodes of the blockchain system 202 based on the model training task for the initial model issued by the device 201 to the blockchain system 202. It is understood that in the solutions provided in the embodiments of this specification, the blockchain system can be initialized based on the model training task.

[0039] Below, in conjunction with Figure 3 This paper introduces the process of initializing a blockchain system based on a model training task. Specifically, Figure 3This is a schematic diagram of the initialization process of the blockchain system in the embodiments of this specification.

[0040] like Figure 3 As shown, firstly, in step S301, the device 201 of the model user sends transaction Tx1 to the blockchain system 202. Transaction Tx1 includes model training task T and initial model IM.

[0041] The model user can be a participant in the blockchain system 202, and the device 201 can be a node in the blockchain system 202; or, the model user can be an ordinary user who can access the blockchain system 202, in which case the device 201 may not be a node in the blockchain system 202.

[0042] The model training task T can, for example, describe the intended use of the model by the user. The user possesses labeled data for multiple samples corresponding to the initial model IM. In one example, these multiple samples represent multiple users. This labeled data can involve several category labels and include training labels corresponding to each sample, indicating the category to which the sample belongs. For example, if the user is a bank, these category labels could include, for example, "has a loan," "has no loan," or "risky," "no risk," etc. The model training task T can, for example, describe the information the bank needs from the model to identify whether a user is risky (has a loan). Similarly, if the user is a product recommender, these category labels could include several product category labels, and the model training task T can, for example, describe the information the product recommender needs from the model for product recommendations.

[0043] In step S303, the blockchain system 202 stores the model training task T and the initial model IM into the blockchain system 202 based on transaction Tx1.

[0044] Specifically, the blockchain system 202 can determine whether to accept the model training task T based on transaction Tx1, and if the model training task T is accepted, store the model training task T and the initial model IM in the blockchain system 202. In one implementation, transaction Tx1 calls a smart contract. When a node in the blockchain system 202 executes transaction Tx1, it stores the model training task T and the initial model IM in the contract state of the smart contract. Afterwards, consensus nodes in the blockchain system 202 can send transactions to the blockchain system 202 to call the smart contract, recording whether they have accepted the model training task T. When a node in the blockchain system 202 executes each of these transactions, it records the number of nodes that have accepted the task in the contract state, and if the number of accepted nodes exceeds a preset number, it records in the contract state that the model training task T has been accepted.

[0045] It should be noted that after transaction Tx1 is executed, the participants in the blockchain system 202 can determine whether their feature data is suitable as training data for this model training based on the model training task T, such as the purpose of the model required by the model user described in the model training task T. When it is determined that the feature data is suitable as training data for this model training and they want to participate in the execution of the model training task T, they can send transaction Tx2 to the blockchain system 202. Transaction Tx2 is used to request participation in the execution of the model training task T.

[0046] Taking nodes 3-5 in blockchain system 202 as an example, when the participants to which these three nodes belong determine that their feature data is suitable as training data for this model training and want to participate in the execution of model training task T, these three nodes can send transaction Tx2 to blockchain system 202.

[0047] In step S305, after transaction Tx1 is executed, blockchain system 202 receives transaction Tx2 sent by nodes of multiple participating parties of blockchain system 202. Transaction Tx2 is used to request participation in the execution of model training task T.

[0048] In one implementation, when transaction Tx2 calls a smart contract, the nodes in the blockchain system 202 can record the node identifiers of the nodes of the aforementioned multiple participants in the node queue of the contract state of the smart contract when executing transaction Tx2.

[0049] In step S307, the blockchain system 202 determines multiple first nodes from the nodes in the blockchain system 202 other than the nodes of the aforementioned multiple participants.

[0050] It should be noted that the first node can be referred to as a committee node. A committee node refers to a representative elected by the participants in the blockchain system, responsible for the operation and maintenance of the blockchain system. These representatives are generally nodes from institutions among the participants that possess authoritative, experienced, and professional characteristics. They can form a committee, which is the highest decision-making body for the autonomous blockchain system. The committee's responsibilities may include joint discussion and decision-making. The committee can assign various roles and tasks according to the needs of the scenario, maintaining the security and stable operation of the blockchain system, ensuring the correctness of the consensus mechanism, and coordinating communication and cooperation among the participants. In the scheme provided in the embodiments of this specification, the committee is mainly responsible for reaching a consensus on the reliability of the intermediate results provided by the training nodes and aggregating intermediate results with high reliability.

[0051] After receiving transaction Tx2 from the nodes of the aforementioned multiple participating parties, blockchain system 202 can identify at least some of the nodes in blockchain system 202 other than the nodes of the aforementioned multiple participating parties as first nodes, and each first node is responsible for identifying multiple second nodes from the nodes of the aforementioned multiple participating parties to participate in the execution of model training task T.

[0052] In one implementation, a node (e.g., a master node) in the blockchain system 202 can select multiple first nodes from among the nodes of the participating parties in the blockchain system 202, and generate a transaction that invokes the smart contract. This transaction includes the node identifiers of the multiple first nodes. When executing the transaction, the blockchain system 202 can store the node identifiers of the multiple first nodes in association with the model training task T in the contract state of the smart contract.

[0053] Taking the nodes of the aforementioned participating parties, including nodes 3-5, as an example, multiple first nodes can be selected from other nodes in the blockchain system 202, such as nodes 1, 2, and 6-8, based on a preset first node selection strategy. The first node selection strategy can be configured according to actual needs and is not specifically limited here.

[0054] In step S309, the aforementioned multiple first nodes determine multiple second nodes from the nodes of the aforementioned multiple participants based on the model training task T and the transaction Tx2 sent by the nodes of the aforementioned multiple participants.

[0055] It should be noted that the second node can be called a training node, and transaction Tx2 can be specifically used to register as a training node to participate in the execution of model training task T.

[0056] As an example, the aforementioned multiple first nodes can obtain information about their designation as first nodes from the contract state, and then proceed to determine the second nodes. When determining the second nodes, the aforementioned multiple first nodes can obtain the identifiers of each node from the node queue in the contract state. Based on these identifiers, they can determine that the nodes of the aforementioned participating parties are requesting to participate in the execution of model training task T. Then, based on the transaction Tx2 sent by the nodes of the aforementioned participating parties, they can obtain the descriptive information related to model training for each of the aforementioned participating parties. This descriptive information may include, for example, business description information and / or data description information for feature data; for example, this descriptive information can be obtained from blockchain system 202 based on the sending account of transaction Tx2, or it can be obtained directly from transaction Tx2. Afterwards, the aforementioned multiple first nodes can determine whether the descriptive information of each of the aforementioned participating parties satisfies model training task T, and if it is determined that the descriptive information satisfies model training task T, the node of that participating party is determined as the second node.

[0057] In one embodiment, after determining multiple second nodes from the nodes of the participating parties, the plurality of first nodes can further execute step S311 to configure a key pair for each determined second node and send the private key of the key pair to the second node. Additionally, the plurality of first nodes can retain the public key of the key pair. Therefore, during the execution of the model training task T, to ensure the security of transmitted data and avoid privacy leaks, the second node can use the private key to encrypt and decrypt transmitted data, and the plurality of first nodes can use the public key to encrypt and decrypt transmitted data with the second node.

[0058] In one implementation, after identifying multiple second nodes from the nodes of the aforementioned participating parties, the first node can send a transaction to the blockchain system 202 to invoke a smart contract. This transaction includes the node identifiers of the multiple second nodes. When executing the transaction, the blockchain system 202 can store the node identifiers of the multiple second nodes in association with the model training task T in the contract state of the smart contract.

[0059] In one implementation, to alleviate the computational burden on the blockchain system 202, a target node from among the multiple first nodes can determine multiple second nodes based on the model training task T and the transactions Tx2 sent by the nodes of the multiple participating parties. The target node can be randomly selected from the multiple first nodes, for example, by one of the multiple first nodes (such as the master first node). Furthermore, after determining the multiple second nodes, the target node can configure key pairs for each of the multiple second nodes and send the node identifier and key pair of each of the multiple second nodes to the other nodes among the multiple first nodes.

[0060] Figure 3 The corresponding implementation provides a solution that allows the blockchain system to determine multiple first nodes and multiple second nodes corresponding to each model training task after receiving it. These second nodes then participate in executing the model training task, and the first nodes aggregate the model training results provided by the second nodes. By re-determining the first and second nodes after each model training task, the security and reliability of the model training can be guaranteed.

[0061] In passing Figure 3 The initialization process described in the corresponding embodiment completes the initialization of the blockchain system 202, and before the aforementioned multiple second nodes perform model training, for example, specifically during the execution of... Figure 4Before the model training initialization process shown, the aforementioned multiple second nodes can work with the device 201 of the model user to determine multiple samples by performing sample alignment. The aforementioned multiple second nodes store feature data of different dimensions of the multiple samples, and the device 201 stores label data Y of the multiple samples. Thus, the device 201 can send the label data Y of the multiple samples to the aforementioned multiple first nodes. The aforementioned multiple second nodes can use the feature data of some dimensions of the multiple samples stored locally to perform model training.

[0062] In one implementation, the aforementioned plurality of second nodes and the device 201 of the model user each store a sample set. During sample alignment, the aforementioned plurality of second nodes can work with device 201 to perform Private Set Intersection (PSI) on each sample set, identifying at least a portion of the samples in the resulting intersection as the aforementioned plurality of samples. Private Set Intersection is a cryptographic technique in secure multi-party computation that allows the two parties involved in the computation to calculate the intersection of their data without obtaining additional information from each other (information other than the intersection). It should be noted that sample alignment technology is a well-known technique in the art. Besides using Private Set Intersection to achieve sample alignment, other well-known techniques can also be used, which will not be elaborated upon here.

[0063] In passing Figure 3 After the initialization process described in the corresponding embodiment completes the initialization of the blockchain system 202, it can execute as follows: Figure 4 The model training initialization process is shown. Figure 4 This is a schematic diagram of the model training initialization process in the embodiments of this specification.

[0064] like Figure 4 As shown, firstly, in step S401, the device 201 of the model user provides label data Y of multiple samples to the aforementioned multiple first nodes.

[0065] The multiple samples can be determined by aligning the aforementioned multiple second nodes with the device 201 used by the model user. The label data Y can involve several category labels and include the training labels corresponding to the multiple samples, which can indicate the category label to which the corresponding sample belongs among the several category labels.

[0066] In step S403, the aforementioned multiple first nodes acquire the initial model IM provided by device 201 as the target model.

[0067] In one example, the aforementioned multiple first nodes can query the initial model IM from the contract state and use the queried initial model IM as the target model. The target model can also be called the global model, and the aforementioned multiple first nodes can be used to update the parameters of this global model.

[0068] In step S405, the aforementioned multiple first nodes send the initial model IM to the aforementioned multiple second nodes.

[0069] In step S407, the aforementioned multiple second nodes determine the initial model IM as the local model.

[0070] exist Figure 4 In the corresponding embodiment, the device 201 of the model user can provide label data Y of multiple samples to the multiple first nodes. Then, the multiple first nodes obtain the initial model IM provided by the device 201 as the target model and send the initial model IM to the multiple second nodes so that the multiple second nodes can use the received initial model IM as the local model for model training. This can achieve model training initialization.

[0071] After model training initialization is complete, the aforementioned multiple second nodes can then be used for model training. Below, we will combine... Figure 5 This section introduces the model training process. Figure 5 This is a timing diagram of a privacy-preserving decentralized longitudinal federated learning model training method in the embodiments of this specification.

[0072] like Figure 5 As shown, firstly, in step S501, the aforementioned multiple second nodes perform forward propagation training on the local model based on the feature data of multiple samples stored locally to obtain intermediate results.

[0073] The multiple samples can be determined by aligning the aforementioned multiple second nodes with the device 201 of the model user.

[0074] As an example, each second node can input the feature data of the aforementioned multiple samples stored locally into its local model to obtain the prediction result output by the local model, and use this prediction result as an intermediate result. It should be noted that the label data Y mentioned above may involve several category labels, and the intermediate result may include sub-results corresponding to the aforementioned multiple samples, which can indicate the category label to which the corresponding sample belongs in those several category labels.

[0075] In step S503, the aforementioned plurality of second nodes provide intermediate results to the aforementioned plurality of first nodes.

[0076] Specifically, the aforementioned multiple second nodes can provide the aforementioned multiple first nodes with plaintext intermediate results; or, in order to ensure the security of transmitted data and avoid privacy leaks, each second node can use a private key (such as the private key in the key pair configured by the aforementioned multiple first nodes for the second node) to encrypt the obtained intermediate results into ciphertext results and provide the ciphertext results to the aforementioned multiple first nodes.

[0077] In one implementation, the blockchain system 202 may deploy a smart contract, which the plurality of second nodes can invoke to provide intermediate results to the plurality of first nodes. For example, the plurality of second nodes can send a transaction invoking the smart contract to the plurality of first nodes, the transaction including the intermediate result or a ciphertext result corresponding to the intermediate result.

[0078] Subsequently, the aforementioned multiple first nodes can obtain the intermediate results provided by the aforementioned multiple second nodes. It should be noted that, when the aforementioned multiple second nodes provide ciphertext results, the aforementioned multiple first nodes can use the public key of each second node (such as the public key in the key pair configured by the aforementioned multiple first nodes for that second node) to decrypt the ciphertext results provided by the second node into plaintext intermediate results.

[0079] After obtaining the intermediate results provided by the multiple second nodes, the multiple first nodes can verify and aggregate the intermediate results by executing steps S505-S511. During this process, the multiple first nodes can ensure the credibility of the second nodes by judging the reliability of the intermediate results, and calculate the global model loss function for intermediate results that meet the conditions.

[0080] Specifically, in step S505, the aforementioned multiple first nodes determine the scores corresponding to each intermediate result for characterizing reliability based on the label data Y, and select multiple first intermediate results whose scores reach the threshold from each intermediate result.

[0081] In this system, the multiple first nodes can use an exclusion method to derive the score of each intermediate result provided by the second node. As an example, assuming the multiple first nodes are reliable, to effectively shorten computation time and improve efficiency, each intermediate result can be scored by at least one first node. Specifically, a target node among the multiple first nodes can determine the first node to score each intermediate result and send the determined result to the other nodes. Then, the multiple first nodes can determine a prediction loss value Loss1 based on the aggregated results of each intermediate result and the label data Y, and a prediction loss value Loss2 based on the aggregated results of intermediate results other than the second intermediate result they are responsible for scoring, and the label data Y. The difference between the prediction loss value Loss2 and the prediction loss value Loss1 is then determined as the score corresponding to the second intermediate result. Next, the target node can receive the scores corresponding to other intermediate results sent by the other nodes. Then, the target node can select multiple first intermediate results whose scores reach a threshold. It should be understood that a first intermediate result can be understood as an intermediate result whose score reaches a threshold.

[0082] In one implementation, aggregation can be achieved by summing the intermediate results to be aggregated. For example, when determining the prediction loss value Loss1, the sum of each intermediate result can be calculated, and this sum can be used as the aggregated result. Then, the difference between this aggregated result and the label data Y can be calculated, and the square of this difference can be used as Loss1. It should be noted that the calculation method for Loss2 is similar to that of Loss1. This includes the intermediate results RL. A Intermediate Results RL B and intermediate results RL c For example, for those responsible for RL of intermediate results B The first node for scoring can calculate Loss1 = (RL) A +RL B +RL C -Y) 2 Loss2 = (RL A +RL C -Y) 2 And use Loss2-Loss1 to obtain the intermediate result RL B The corresponding score.

[0083] In step S507, the aforementioned multiple first nodes determine the prediction loss value Loss3 of the target model based on the aggregation result of the aforementioned multiple first intermediate results and the label data Y.

[0084] One approach is similar to the prediction loss calculation method described above. Based on the aggregation result of the multiple first intermediate results and the label data Y, the prediction loss value Loss3 of the target model can be determined. For example, the sum of the multiple first intermediate results can be calculated, and this sum can be used as the aggregation result of the multiple first intermediate results. Then, the difference between this aggregation result and the label data Y can be calculated, and the square of this difference can be calculated and used as the prediction loss value Loss3.

[0085] In one implementation, when there are intermediate results whose scores do not reach a threshold, i.e., when there are unreliable intermediate results, the aforementioned multiple first nodes can record the number of times the second node providing the unreliable intermediate result has failed to meet the required standards. Upon reaching a preset number of failures, the first nodes are determined to be unreliable, preventing the unreliable intermediate result from participating in the final result aggregation, and thus proceeding to step S507. Furthermore, the aforementioned multiple first nodes can also record the second node as an unreliable node, thereby preventing the second node from performing subsequent model training processes.

[0086] In step S509, in response to the prediction loss value Loss3 not satisfying the model convergence condition, the plurality of first nodes determine the gradient values ​​corresponding to the plurality of third nodes based on the prediction loss value Loss3 and the current first model parameters of the local models of the plurality of third nodes that provide the plurality of first intermediate results among the plurality of second nodes.

[0087] The convergence condition of the model may include, for example, the accuracy of the trained model. The gradient values ​​corresponding to the multiple third nodes can be determined using the backpropagation algorithm.

[0088] As an example, the aforementioned multiple first nodes can determine their respective gradient values ​​based on the backpropagation algorithm, using the target model's loss function L3 for calculating Loss3, the predicted loss value Loss3, and the current first model parameters of their respective local models. Assume the aforementioned multiple third nodes include node A and node B, and node A submits the first intermediate result RL. A Node B submits the first intermediate result RL B , where RL A =F A (w A X A ), RL B =F B (w B X B In this case, L3 = (F A (w A X A )+FB (w B X B )-Y) 2 Among them, w i This represents the first model parameter of the local model of node i, specifically the function F corresponding to that local model. i Model parameters of (); X i This represents the feature data of the above multiple samples stored locally at node i, and its function is F. i The input data is ( ); i is one of A and B. The gradient value corresponding to node i can be expressed as in, The symbol represents the partial derivative.

[0089] In one implementation, in order to reduce the computational pressure on the blockchain system 202, after the target node selects multiple first intermediate results whose scores reach the threshold from the various intermediate results, the target node can then determine the predicted loss value Loss3 of the target model by executing step S507, and when it is determined that the model convergence condition is not met based on the predicted loss value Loss3, the gradient values ​​corresponding to the multiple third nodes are determined by executing step S509.

[0090] In step S511, the aforementioned first nodes send their corresponding gradient values ​​to the aforementioned third nodes respectively.

[0091] Specifically, the aforementioned first nodes can send plaintext gradient values ​​to the aforementioned third nodes; or, in order to ensure the security of transmitted data and avoid privacy leaks, the aforementioned first nodes can use the public key of each third node to encrypt the gradient value corresponding to the third node into a ciphertext gradient value, and send the ciphertext gradient value to the third node.

[0092] Subsequently, the aforementioned multiple third nodes can obtain their respective gradient values. It should be noted that when each third node receives the ciphertext gradient value, it can use its private key to decrypt the ciphertext gradient value into the plaintext gradient value.

[0093] After the aforementioned third nodes obtain their respective gradient values, the aforementioned third nodes can perform backpropagation training on the local model by executing step S513.

[0094] In step S513, the aforementioned third nodes update the parameters of the local model based on the gradient values.

[0095] Taking node i as an example, as described above, node i can be represented by, for example, the formula... Update the corresponding model parameters. Among them, w i ' can represent the second model parameter of the local model update of node i, and η can represent the learning rate.

[0096] In step S515, the aforementioned third nodes provide the aforementioned first nodes with second model parameters for local model updates.

[0097] Specifically, the aforementioned third nodes may provide the aforementioned first nodes with plaintext second model parameters; or, in order to ensure the security of transmitted data and avoid privacy leaks, each third node may use its private key to encrypt the second model parameters into ciphertext model parameters and provide the ciphertext model parameters to the aforementioned first nodes.

[0098] Subsequently, the aforementioned multiple first nodes can obtain the second model parameters provided by the aforementioned multiple third nodes. It should be noted that, when the aforementioned multiple third nodes provide ciphertext model parameters, the aforementioned multiple first nodes can use the public key of each third node to decrypt the ciphertext model parameters provided by the third node into the second model parameters.

[0099] In step S517, the aforementioned multiple first nodes determine the model performance of the target model based on each second model parameter.

[0100] The model performance can include any one of the following performance parameters: accuracy, prediction loss, error rate, precision, and recall.

[0101] In one implementation, the model performance of the target model includes prediction loss. The aforementioned plurality of first nodes may store feature data X of several test samples T. T and label data Y T Label data Y T The label data Y described earlier involves the same category labels. Specifically, the label data Y... T This includes the training label corresponding to each test sample in the plurality of test samples T, and the training label indicates the corresponding test sample in the label data Y. T The category label to which each category label belongs. In this case, the aforementioned multiple first nodes can be based on each second model parameter and feature data X. T Calculate intermediate results RL T Based on intermediate results RL T and label data Y T Calculate the prediction loss value Loss4 of the target model. Continuing with the example of multiple third nodes including node A and node B, assume that node A provides the second model parameter w. A Node B provides the second model parameter w′. B The aforementioned multiple first nodes can be used to calculate RL. T =w A ′×X T +w′ B ×XT Loss4 = (RL T -Y T ) 2 .

[0102] In one implementation, the model performance of the target model includes prediction loss. When the aforementioned plurality of second nodes perform sample alignment with the device 201 of the model user, in addition to determining the aforementioned plurality of samples, they can also determine a plurality of test samples T. For example, a portion of the samples in the intersection as described above can be determined as the aforementioned plurality of samples, and the remaining samples can be determined as the plurality of test samples T. In addition to providing the aforementioned plurality of first nodes with the label data Y of the aforementioned plurality of samples, the device 201 can also provide the aforementioned plurality of first nodes with the label data Y of the plurality of test samples T. T In this scenario, after executing step S513, the aforementioned third nodes can use the feature data of the locally stored test samples T to perform forward propagation training on the updated local model to obtain intermediate results, and then provide these intermediate results to the aforementioned first nodes. Subsequently, the aforementioned first nodes can aggregate the various intermediate results associated with the test samples T to obtain an aggregated result, and based on this aggregated result and the label data Y... T Determine the prediction loss value Loss4 for the target model.

[0103] In one implementation, if the aforementioned plurality of first nodes are trusted, in order to reduce the computational pressure on the blockchain system 202, one of the aforementioned first nodes can determine the prediction loss value Loss4 of the target model by executing step S517.

[0104] In step S519, the aforementioned multiple first nodes, in response to the model performance satisfying the model convergence condition, determine each second model parameter as the updated parameter of the target model.

[0105] Specifically, when the target model's performance includes the predicted loss value Loss4 as described above, the aforementioned multiple first nodes can determine each second model parameter as the updated parameter of the target model in response to the predicted loss value Loss4 satisfying the model convergence condition.

[0106] In one implementation, after step S519, the aforementioned plurality of first nodes can store each of the second model parameters in the blockchain system 202. Further, the aforementioned plurality of first nodes can package each of the second model parameters into a block and store the block in the blockchain system 202.

[0107] In one implementation, when the target model's performance does not meet the model convergence condition, the aforementioned first nodes can also send feedback information to the aforementioned third nodes, instructing them to perform the next round of model training on the local model. Thus, the aforementioned third nodes can perform actions such as... Figure 5 The model training process is shown.

[0108] Figure 5 The corresponding embodiment provides a solution whereby the intermediate results are obtained by performing forward propagation training on the local model using the feature data of the multiple samples stored locally by the multiple second nodes, and then providing the intermediate results to the multiple first nodes. This ensures that the feature data does not leave the domain, thereby guaranteeing data security and preventing privacy leaks. The solution involves the multiple first nodes determining a reliability score for each intermediate result, selecting multiple intermediate results with scores reaching a threshold, and determining the target model's prediction loss value Loss3 based on the aggregation result of the multiple intermediate results and the label data Y. Then, based on the prediction loss value Loss3 and the current first model parameters of the local models of the multiple third nodes that provided the multiple intermediate results, the solution determines the gradient values ​​corresponding to each of the multiple third nodes and sends these gradient values ​​to them. This allows the multiple third nodes to update their local model parameters based on the obtained gradient values. Intermediate results with scores below the threshold are considered unreliable (invalid), preventing them from participating in the final aggregation process and preventing the second nodes providing these intermediate results from performing subsequent model training, thus providing good reliability for the target model. Model aggregation is achieved by using multiple first nodes rather than a single server, which strengthens the resistance to single-point attacks and single-point failures, and effectively prevents the leakage of intermediate results and training data. Therefore, this scheme enables multiple participants with the same sample space but different feature spaces to perform the same model training task, while ensuring data security, preventing privacy leaks, and guaranteeing the reliability of the target model being trained.

[0109] As described above, the aforementioned multiple first nodes and multiple second nodes can be included in a blockchain system, such as a consortium blockchain system. The solution provided in the embodiments of this specification can implement a decentralized vertical federated learning model training method based on a consortium blockchain, which can achieve the following technical effects:

[0110] 1. This method offers strong security. Specifically, it is based on a decentralized consortium blockchain rather than a single server, making it highly resistant to single points of attack and failure, effectively preventing the leakage of training results and process data. Furthermore, the consortium blockchain provides an effective access control mechanism, excluding malicious nodes from the blockchain and preventing them from interfering with the federated learning process or even stealing training data.

[0111] 2. This method exhibits good reliability. Specifically, the committee consensus mechanism in this method can exclude invalid data training results from the final aggregation process, thus providing good reliability for the global model.

[0112] Below, we will take the blockchain system described above as an example. Figure 6 The example shown is a consortium blockchain, which includes nodes from multiple participating parties such as a bank (Party A), an online shopping platform (Party B), and a social media platform (Party C). It is used in conjunction with... Figure 6 and Figure 7 The solutions provided in the embodiments of this specification are illustrated with examples. Figure 6 and Figure 7 These are all schematic diagrams illustrating the decentralized vertical federated learning model training method based on consortium blockchain in the embodiments of this specification.

[0113] This example illustrates a multi-party collaboration in the sales industry to achieve a comprehensive analysis of consumer demand and purchasing power. The bank (Party A), the online shopping platform (Party B), and the social media platform (Party C) collaborate on information sharing, and the characteristic data held by each party can be stored in their respective local databases. For example, Figure 7 As shown, Party B can hold feature data of dimension X3 for four samples A2, A3, A4, and A5, as well as label data Y corresponding to these four samples, while Party A and Party C only hold feature data and not label data. Specifically, Party A holds feature data of dimensions X1 and X2 for three samples A1, A2, and A4, while Party C holds feature data of dimensions X4, X5, and X6 for three samples A1, A2, and A4. In one example, the dimensions in this embodiment can be referred to as attributes. It should be understood that the six dimensions X1-X6 are all different, and Party A, Party B, and Party C have the same sample space but different feature spaces. To protect data security and prevent privacy leaks, the feature data of each party will not be aggregated to one party for related training. Each party uses the model training method based on consortium blockchain provided in this embodiment to train the model based on its own feature data and derive a complete global model to analyze user needs and achieve accurate product recommendations.

[0114] Specifically, when an online shopping platform (Party B) has a product recommendation need, Party B can act as a model user, using its nodes to publish a model training task and an initial model for federated learning to the consortium blockchain. For example, it can send a transaction Tx1 to the consortium blockchain including the task and the initial model. The consortium blockchain can then decide whether to accept the task. If it does not accept the task, the consortium blockchain can return feedback information to Party B's nodes indicating its refusal to execute the task and terminate the processing of the task. If it accepts the task, Parties A, B, and C can request to register as training nodes participating in the task through their respective nodes, for example, by sending a transaction Tx2 to the consortium blockchain to register as training nodes participating in the task. After receiving transaction Tx2 from the nodes of Parties A, B, and C, the consortium blockchain can proceed as follows... Figure 6 As shown, a committee is formed by selecting multiple nodes from the consortium blockchain, excluding those of parties A, B, and C. These committee nodes are used for processing model training results, such as discrimination and aggregation. It should be noted that the committee's responsibilities may include joint discussion and decision-making (e.g.,...) Figure 7 The diagram illustrates the decision to make a YES or NO decision. After the committee is formed, each committee node can determine multiple training nodes from the nodes of parties A, B, and C to participate in model training. For example, if nodes from parties A, B, and C are all determined as training nodes, then training nodes for parties A, B, and C can be generated. Next, each committee node can configure a key pair (SK) for the training nodes of party A. A PK A Configure key pairs (SK) for the training nodes of Party B. B PK B Configure key pairs (SK) for the C-side training nodes. C PK C It sends the private key from its corresponding key pair to each training node and retains the public key of each training node. Among them, SK i This represents the private key of the training node i, PK. i Let i represent the public key of the training node i, where i is one of A, B, and C.

[0115] Next, the training nodes of party A, party B, and party C can be trained as follows: Figure 7 As shown, multiple samples, such as samples A2 and A4, are determined by sample alignment. The B-side training node locally stores the feature data and label data Y of these multiple samples, while the A-side and C-side training nodes locally store the feature data of these multiple samples.

[0116] Next, the training node of Party B can provide the labeled data Y for model training to each committee node. Each committee node can respond to receiving the labeled data Y by, as follows: Figure 6 As shown, the initial model provided by the node of party B is obtained as the global model, and the initial model is sent to the training nodes of party A, party B and party C so that these three training nodes can use the initial model as the local model for model training.

[0117] During model training, each training node can perform forward propagation training on its local model based on the feature data of locally stored samples A2 and A4 to obtain intermediate results. For example... Figure 7 As shown, training node A can perform forward propagation training on the local model (Local model A) based on the feature data of samples A2 and A4 stored locally, and obtain the intermediate result RL. A The training node B can perform forward propagation training on the local model (Local model B) based on the feature data of locally stored samples A2 and A4, and obtain the intermediate result RL. B The C-side training node can perform forward propagation training on the local model (Local model C) based on the feature data of locally stored samples A2 and A4, obtaining the intermediate result RL. C Among them, RL i =F i (w i X i ), w i This represents the current model parameters of the local model of training node i, specifically the function F corresponding to that local model. i Model parameters of (); X i This represents the feature data of samples A2 and A4 stored locally on the i-th training node, and its function is F. i The input data is ( ); i is one of A, B, and C.

[0118] Next, each training node can use the private key distributed by the committee node to encrypt the intermediate results into ciphertext. Here, the ciphertext results of training nodes A, B, and C are denoted as [RL] respectively. A ],[RL B ],[RL C Then, each training node can provide the encrypted results to each committee node. In one example, a smart contract can be deployed in the consortium blockchain, which each training node can call to provide the encrypted results to each committee node.

[0119] After receiving the encrypted result, each committee node can use the public key from the key pair configured for each training node to decrypt the encrypted result provided by that training node into an intermediate result, thereby obtaining the intermediate result RL. A ,RL B ,RL C .

[0120] Next, each committee node can determine the reliability of these three intermediate results and select the intermediate result with higher reliability for aggregation. For example, an exclusive method can be used to obtain the score of the intermediate result for each training node, ensuring that each intermediate result is determined by at least one committee node.

[0121] As an example, a target node in each committee node can determine the committee node responsible for scoring each intermediate result and send the determination to the other committee nodes. Then, each committee node can determine a prediction loss value Loss1 based on the aggregated results of each intermediate result and the label data Y, and a prediction loss value Loss2 based on the aggregated results of intermediate results other than the second intermediate result it is responsible for scoring, and the label data Y. The difference between prediction loss value Loss2 and prediction loss value Loss1 is then determined as the score corresponding to the second intermediate result. (The text then abruptly shifts to a different topic: "Intermediate results RL trained on side B.") B For example, the committee node responsible for scoring this intermediate result can calculate the prediction loss value Loss1 = (RL A +RL B +RL C -Y) 2 Then, assign your own intermediate scores (RL) B Exclude it and recalculate the predicted loss value. For example, calculate the predicted loss value Loss2 = (RL) A +RL C -Y) 2 Then, the intermediate result RL is obtained by subtracting Loss2 from Loss1. B The corresponding score. A higher score indicates a more reliable intermediate result. Next, the target node can receive scores corresponding to other intermediate results sent by the other committees and select multiple first intermediate results whose scores reach a threshold. Furthermore, the target node can send the selection results to the other committee nodes. It should be noted that, in the following text, these multiple first intermediate results include RL. A and RL B Let's take an example to describe it.

[0122] Next, each committee node can base its decision on the first intermediate result RL. A RL BThe aggregated results and label data Y are used to determine the current prediction loss value Loss3 of the global model. For example, prediction loss value Loss3 = (RL) A +RL B -Y) 2 Then, each committee node can determine whether the model convergence condition is met based on the predicted loss value Loss3. This convergence condition may include, for example, the accuracy of the trained model. If not, each committee node can determine the gradient value corresponding to each training node based on the backpropagation algorithm, using the global model's loss function L3 used to calculate Loss3, the predicted loss value Loss3, and the current model parameters of each training node's local model. Specifically, L3 = (F... A (w A X A )+F B (w B X B )-Y) 2 Here, the gradient values ​​corresponding to the training nodes of side A and side B are denoted as follows:

[0123] Next, each committee node can use the public key from the key pair configured for training node i to encrypt the gradient value corresponding to training node i into a ciphertext gradient value, where i is one of A and B. Here, the ciphertext gradient values ​​corresponding to training nodes A and B are denoted as follows: Afterwards, each committee node can send the encrypted gradient values ​​to the training node of Party A. And sending encrypted gradient values ​​to the training node of Party B.

[0124] Next, training nodes A and B can use the private key distributed by the committee node to decrypt the obtained ciphertext gradient value into a plaintext gradient value. Thus, training node A can obtain the plaintext gradient value. The training nodes of side B can obtain the gradient values ​​of the plaintext. Afterwards, training nodes A and B can update the parameters of their local models based on the plaintext gradient values. Here, the updated model parameters of each training node's local model are denoted as w. A ′,w′ B Then, the training nodes of Party A and Party B can provide updated model parameters to the committee nodes.

[0125] Next, each committee node can base its decisions on w. A ′,w′ BFirst, determine the current predicted loss value Loss4 for the global model. The calculation method for Loss4 can be found in the previous explanation and will not be repeated here. Then, each committee node can determine whether the model convergence condition is met based on the predicted loss value Loss4. If it is met, each committee node can then... A ′,w′ B The parameters to be updated for the global model are determined, and as follows: Figure 6 and Figure 7 As shown, the updated parameters of the global model are written into the consortium blockchain; where, Figure 7 The Blockchain shown can represent a consortium blockchain. If this condition is not met, each committee node can send feedback information to the A and B training nodes, instructing them to perform the next round of model training on their local model. Upon receiving this feedback, the A and B training nodes can then perform the next round of model training on their current local model.

[0126] Figure 8 This is a schematic diagram of the structure of a privacy-preserving decentralized longitudinal federated learning model training device as described in the embodiments of this specification. This device can be applied to one of a plurality of first nodes, which store label data corresponding to multiple samples of the target model. The device can perform actions such as... Figures 3 to 5 The steps are performed by the first node. The device includes: an acquisition unit 801, configured to acquire intermediate results provided by a plurality of second nodes, the intermediate results being obtained by the second nodes performing forward propagation training on the local model corresponding to the target model based on the feature data of the plurality of samples stored locally, the plurality of second nodes storing feature data of the plurality of samples in different dimensions; a selection unit 802, configured to determine a score for representing reliability corresponding to each intermediate result based on the label data, and select a plurality of first intermediate results whose scores reach a threshold from the plurality of intermediate results; a loss determination unit 803, configured to determine a first predicted loss value of the target model based on the aggregation result of the plurality of first intermediate results and the label data; a gradient determination unit 804, configured to determine the gradient value corresponding to each of the plurality of third nodes based on the first predicted loss value and the current first model parameters of the local models of the plurality of third nodes that provide the plurality of first intermediate results, in response to the first predicted loss value not satisfying the model convergence condition; and a sending unit 805, configured to send the corresponding gradient value to the third node among the plurality of third nodes.

[0127] This specification also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, it causes the computer to perform actions such as... Figures 3 to 5 The method shown in any of the above.

[0128] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements, as shown in the embodiment. Figures 3 to 5 The method shown in any of the above.

[0129] This specification also provides a computer program product, wherein when the computer program product is executed in a computer, it causes the computer to perform the following: Figures 3 to 5 The method shown in any of the above.

[0130] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0131] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0132] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0133] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0134] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0135] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0139] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0140] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0141] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0143] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0144] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A privacy-preserving decentralized longitudinal federated learning model training method, executed by multiple first nodes, wherein the multiple first nodes store label data corresponding to multiple samples of the target model, the method comprising: The intermediate results provided by multiple second nodes are obtained. The intermediate results are obtained by the second nodes performing forward propagation training on the local model corresponding to the target model based on the feature data of the multiple samples stored locally. The multiple second nodes store feature data of the multiple samples in different dimensions. Based on the label data, a score corresponding to each intermediate result is determined to characterize reliability, and multiple first intermediate results whose scores reach a threshold are selected from the intermediate results. Based on the aggregation result of the multiple first intermediate results and the label data, the first prediction loss value of the target model is determined; In response to the first predicted loss value not satisfying the model convergence condition, the gradient values ​​corresponding to the multiple third nodes are determined based on the first predicted loss value and the current first model parameters of the local models of the multiple third nodes that provide the multiple first intermediate results among the multiple second nodes; The gradient values ​​are sent to the plurality of third nodes respectively.

2. The method according to claim 1, wherein, The step of determining the reliability score corresponding to each intermediate result based on the label data, and selecting multiple first intermediate results whose scores reach a threshold from the intermediate results, includes: One of the target nodes among the plurality of first nodes determines the first node for scoring each of the intermediate results, and sends the determination result to the other nodes among the plurality of first nodes; The plurality of first nodes determine a second predicted loss value based on the aggregation result of each intermediate result and the label data, and determine a third predicted loss value based on the aggregation result of the intermediate results other than the second intermediate result that it is responsible for scoring and the label data, and determine the difference between the third predicted loss value and the second predicted loss value as the score corresponding to the second intermediate result; The target node receives the scores corresponding to other intermediate results sent by the other nodes; The target node selects multiple first intermediate results from the various intermediate results whose scores reach the threshold.

3. The method according to claim 1, wherein, After sending the corresponding gradient values ​​to the plurality of third nodes respectively, the process further includes: Obtain the second model parameters updated by the local model provided by the plurality of third nodes. The second model parameters are obtained by the third nodes after updating the parameters of the local model based on the obtained gradient values. Based on each of the second model parameters, the model performance of the target model is determined; In response to the model performance satisfying the model convergence condition, each of the second model parameters is determined as the updated parameters of the target model.

4. The method according to claim 3, wherein, The plurality of first nodes and the plurality of second nodes are contained in the blockchain system; as well as After determining that each of the second model parameters is an updated parameter of the target model, the method further includes: Each of the second model parameters is stored in the blockchain system.

5. The method according to claim 3, further comprising: In response to the model performance not meeting the model convergence condition, feedback information is sent to the plurality of third nodes to indicate that the local model should be trained in the next round.

6. The method according to claim 1, wherein, The initial target model is the initial model provided by the device of the model user; as well as Before retrieving the intermediate results provided by multiple second nodes, the process also includes: Obtain the label data provided by the device of the user of the model; The initial model is obtained as the target model; The initial model is sent to the plurality of second nodes so that the plurality of second nodes can use the initial model as a local model for model training.

7. The method according to claim 6, wherein, The plurality of first nodes and the plurality of second nodes are contained in the blockchain system; as well as Before obtaining the label data provided by the device of the model user, the process also includes: The system receives a first transaction sent by nodes of multiple participants in the blockchain system, the first transaction being used to request participation in the execution of a model training task; Based on the model training task for the initial model issued by the device of the model user and the first transaction, the plurality of second nodes are determined from the nodes of the plurality of participating parties.

8. The method according to claim 7, wherein, After determining the plurality of second nodes from the nodes of the plurality of participating parties, the method further includes: Configure a key pair for each identified second node, and send the private key of the key pair to the second node.

9. The method according to claim 8, wherein, The process of obtaining intermediate results provided by multiple second nodes includes: Obtain the ciphertext results provided by the plurality of second nodes, wherein the ciphertext results are obtained by the second nodes encrypting the intermediate results using the private key; Using the public key from the key pair configured for each second node, the ciphertext result provided by the second node is decrypted into an intermediate result.

10. The method according to claim 7, wherein, Each of the first transactions is sent to the blockchain system by the nodes of the plurality of participating parties. The plurality of first nodes are determined by the blockchain system from the nodes other than the nodes of the plurality of participating parties after receiving each of the first transactions.

11. The method according to claim 10, wherein, The model training task and the initial model are included in a second transaction sent from the device of the model user to the blockchain system. The blockchain system stores the model training task and the initial model in the blockchain system based on the second transaction. Each of the first transactions is sent by the nodes of the plurality of participating parties after the second transaction is executed.

12. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-11.

13. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-11.

Images