A decentralized heterogeneous neural network federated learning method and system

Abstract

The application provides a decentralized heterogeneous neural network federated learning method and system, a service end initiates recruitment, registers through a smart contract and provides a bonus pool, a training end responds and provides a deposit; the service end only holds labels, and the training end only holds data, a local model of a complete task front end executing an upstream task is deployed on the training end, a top model of a rear end is deployed on the service end, the training end performs forward propagation on the local model based on local data to obtain intermediate results, the intermediate results are sent to a service end smart contract bridge through a training end smart contract bridge, are aggregated by the service end smart contract bridge, forward propagation is performed on the top model by the service end, loss and gradient information are calculated by using the local labels, backward propagation is performed to update the top model, the loss and the gradient information are sent back to the training end smart contract bridge through the service end smart contract, and backward propagation is performed on the local model by the training end to update the local model. Data islands are eliminated, interaction of model parameters is not needed, model weight leakage is prevented, and user privacy is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a decentralized heterogeneous neural network federated learning method and system. BACKGROUND

[0002] With the rapid development of digital technology, data is growing explosively, and artificial intelligence is rapidly developing to an unprecedented height, bringing new opportunities and challenges for the upgrading and reform of traditional machine learning. On the one hand, it provides direction for the development of traditional machine learning; on the other hand, it also brings new challenges to data and network security. Due to the existence of an insurmountable barrier between data sources, the data required by artificial intelligence generally involves multiple fields, and the data exists in the form of an island, which is one of the key challenges.

[0003] The training of artificial intelligence algorithm model requires a large amount of data, and these data come from various industries. Due to industry competition, privacy security, complex administrative procedures and other problems, the cost required to integrate these data is huge. Google proposed federated machine learning in 2016, which can train models without data sharing. Federated machine learning is a machine learning method, and participants can train the overall model by training sub-models. Participants train the model collaboratively under the coordination of the central server while keeping the training data decentralized. Users train sub-models locally and send them to a trusted third party for model merging. The federated machine learning method eliminates the need to collect all user data, but recent studies have found that there is a risk of exposing model weights to the center node, which is essentially a centralized system and cannot optimize users participating in training. SUMMARY

[0004] In view of this, the embodiments of the present application provide a decentralized heterogeneous neural network federated learning method and system to eliminate or improve one or more defects in the prior art, solve the problem that the existing federated learning easily exposes model weight privacy to the center node during the training process.

[0005] One aspect of the present application provides a decentralized heterogeneous neural network federated learning method, which is based on the joint implementation of multiple participants, including a service end and multiple training ends; the local data samples of the service end and the training end are aligned; the service end only holds labels and is connected to the outside through a service end smart contract bridge, and the training end holds data and is connected to the outside through a training end smart contract bridge; the method comprises the following steps:

[0006] The service end registers with the smart contract, provides service end registration information and a bonus pool to initiate recruitment of the training end; the training end registers with the smart contract in response to the recruitment, provides training end registration information and provides a pledge;

[0007] The training end performs forward propagation on the local model based on the local data sample to generate intermediate results, and sends the intermediate results to the service end through the training end smart contract bridge after aggregation and forwarding to the service end;

[0008] The service end performs forward propagation on the top model based on the aggregated intermediate results, calculates loss and gradient information using the locally held labels, and updates the top model in reverse; the top model is the top part of the complete task, and the local model of each training end is used to perform an upstream task and jointly constitutes the complete task with the top model;

[0009] The service end distributes the loss and the gradient information to the training end smart contract bridge through the service end smart contract bridge, so that each training end performs back propagation update on the corresponding local model until a preset termination condition is reached;

[0010] The service end encrypts the top model and distributes it to multiple training ends as a plurality of continuous sub-models, and provides an initial test input; the output of each sub-model in each training end is uploaded to the smart contract and downloaded and run by the next training end in sequence, and the final test result is returned to the service end;

[0011] The service end calculates the difference in prediction results of each feature in the absence or introduction state in a set number of rounds, averages to obtain the influence score of the feature; the influence scores of the feature values contained in the local data of each training end are summed to obtain the contribution index of the corresponding training end, the bonus pool is distributed according to the contribution index, and the pledge is disposed.

[0012] In some embodiments, the service end and the training end are each provided with an IPFS client to store the intermediate results generated in each round of training to an InterPlanetary File System, based on a blockchain for persistence, and to perform persistence verification on the intermediate results saved in the InterPlanetary File System after each set number of training, the persistence verification including integrity checking, data storage node verification, and storage retrieval reliability verification.

[0013] In some embodiments, the method further comprises: in the process of registration of the service end and the training end to the smart contract, respectively dispatching asymmetric keys for the service end and the training end by the smart contract, broadcasting the public key in the asymmetric keys, the private key in the asymmetric keys being held by the service end and the training end respectively, and using the asymmetric keys held respectively for encrypted communication in the process of transmission of the intermediate result, the loss and the gradient information.

[0014] In some embodiments, the influence score of each feature is obtained by calculating the difference of the prediction results of each feature in the missing or introduced state in a set number of rounds, including:

[0015] Two samples are constructed and , the difference only lies in the feature ;

[0016] The calculation formula of the influence score of the fth feature is:

[0017] ;

[0018] Wherein, the function m represents the prediction result, and N represents the set number.

[0019] In some embodiments, the calculation formula of the contribution degree index of the training end is:

[0020] ;

[0021] Wherein, represents the contribution degree index of the jth training end in the ith round, and F represents the number of features in the local data.

[0022] The contribution degree index is modified,

[0023] ;

[0024] Wherein, represents the modified contribution degree index, represents the accuracy value of the ith training end in the pretest process, and M represents the number of training ends.

[0025] In some embodiments, the bonus pool is distributed according to the contribution degree index, and the disposal of the pledge includes:

[0026] Screening out unqualified training ends with a contribution degree index lower than a preset contribution degree threshold, and deducting the pledge of the unqualified training ends;

[0027] According to the contribution index, the contribution proportion of the training end participating in the training except the unqualified training end is calculated, and the bonus in the bonus pool is distributed to each training end participating in the training according to the contribution proportion; the calculation formula of the contribution proportion is:

[0028]

[0029] wherein, represents the contribution proportion of the i-th client, and M represents the number of the training ends participating in the training, represents the modified contribution index;

[0030] The bonus allocation formula of the j-th training end is:

[0031]

[0032] wherein, Pool represents the bonus pool amount.

[0033] In some embodiments, the method further comprises marking the unqualified training end and limiting its participation in subsequent training.

[0034] In another aspect, the application also provides a decentralized heterogeneous neural network federated learning system, which is implemented by multiple participants, one of the participants acts as a server and the other multiple participants act as training ends; the local data samples of the server and the training ends are aligned; the server only holds labels and is connected to the outside through a server smart contract bridge, and the training ends hold data and are connected to the outside through a training end smart contract bridge; the system executes the above-mentioned decentralized heterogeneous neural network federated learning method.

[0035] In another aspect, the application also provides a computer readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method.

[0036] In another aspect, the application also provides a computer program product comprising a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned method.

[0037] The application has at least the following beneficial effects:

[0038] ​The decentralized heterogeneous neural network federated learning method and system, the server initiates recruitment, registers through a smart contract and provides a bonus pool, and the training end responds and provides a deposit; the server only holds labels, and the training end only holds data, the local model of the complete task front-end executes the upstream task is deployed on the training end, and the top model of the back-end is deployed on the server, the training end performs forward propagation on the local model based on the local data to obtain intermediate results, sends the intermediate results to the server smart contract bridge through the training end smart contract bridge, aggregates the intermediate results by the server smart contract bridge, and then the server performs forward propagation on the top model and calculates loss and gradient information by using the local labels, and then performs backward propagation to update the top model, and sends the loss and gradient information back to the training end smart contract bridge through the server smart contract, and then the training end performs backward propagation to update the local model. In the case of eliminating data islands, the model parameters do not need to be interacted in the training process, the model weight leakage can be effectively prevented, and the user privacy can be ensured.

[0039] Further, the server establishes a bonus pool, and the training end provides a deposit; the top model is divided into a plurality of sub-models, on the basis of providing an initial test input, the sub-models are deployed to the training end for continuous collaborative calculation, the influence scores of the features contained in each data are obtained, and then the contribution degree indexes of each training end are calculated, the bonuses in the bonus pool are allocated based on the contribution degree indexes, and the deposit of the training end that does not meet the requirements is deducted, so as to encourage each training end to provide better training service and optimize the training effect.

[0040] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

[0041] Those skilled in the art will appreciate that the objects and advantages of the application can be realized and attained by means summarized fully in the following written description and claims, and further realized and attained by means of the many embodiments both as specifically described and as apparent to one skilled in the art in view of the written description and claims hereof. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the written description serve to explain the principles of the application. In the drawings:

[0043] Figure 1 The structure diagram of the federated learning system according to an embodiment of the application.

[0044] Figure 2 The flowchart of the decentralized heterogeneous neural network federated learning method according to an embodiment of the application.

[0045] Figure 3 A logic diagram for performing data persistence and verification in the decentralized heterogeneous neural network federated learning method described in an embodiment of the present application.

[0046] Figure 4 A structure diagram for splitting a top model in the decentralized heterogeneous neural network federated learning method described in an embodiment of the present application.

[0047] Figure 5 A logic diagram for calculating SHAP values in the decentralized heterogeneous neural network federated learning method described in an embodiment of the present application. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the present application clearer, further detailed descriptions will be given to the present application in conjunction with the embodiments and drawings. Herein, the illustrative embodiments of the present application and their descriptions are used to explain the present application, but not as a limitation to the present application.

[0049] Herein, it is also necessary to note that, in order to avoid the present application being obscured by unnecessary details, only the structures and / or processing steps closely related to the solutions according to the present application are shown in the drawings, and other details not closely related to the present application are omitted.

[0050] It should be emphasized that the term “comprises / comprising” is used herein to indicate the presence of the stated features, elements, steps, or components, but does not preclude the presence or addition of one or more other features, elements, steps, or components.

[0051] Herein, it is also necessary to note that, if not specifically stated, the term “connection” can not only mean direct connection, but also indirect connection with an intermediate.

[0052] In the decentralized exploration of federated machine learning, there is a relatively good overall solution idea. For example, the decentralization of federated machine learning is realized by means of an election committee, which proposes three model election methods: election committee with accuracy as weight, random election committee, and multi-element election committee, but this method is only suitable for horizontal federated learning scenarios. In the prior art, there is also a vertical federated learning training method with a smart contract as the interaction center, which has slower speed and reduced training performance.

[0053] Specifically, one aspect of the present application provides a decentralized heterogeneous neural network federated learning method, which is based on the joint implementation of a plurality of participants, including a server and a plurality of training ends; the local data samples of the server and the training end are aligned; the server only holds labels and is connected to the outside through a server smart contract bridge, and the training end holds data and is connected to the outside through a training end smart contract bridge.

[0054] Referring Figure 1 In the present application, multiple participants each hold their own local data and / or labels, and are divided into active and passive parties based on different tasks during the training process. When a participant initiates a training task, it acts as an active party and performs training in the role of a server. When other participants respond to the training task, they act as passive parties and perform training in the role of training ends. During the training process, the server only participates in training with the locally held labels, and the training end participates in training with the locally held data. It should be emphasized that the data of the training end during the training process is unlabeled. In order to ensure that the training can be completed, it is required that the samples held by the server and the training end are aligned. For example, two training end data sets A and B both have the same sample with an id of 1, but the A data set has feature columns of name, gender, and age, while the B data set has name, income, and height. Such a case is considered as having the same sample space but different feature spaces for the A and B data sets, and the server holds the label of the sample.

[0055] It should be noted that the data held by the training end can be different in the feature space, and the local model structure executed by the training end and the task executed can be the same or different. The purpose is mainly to expand the task function or expand the same model in the feature domain. The top model executed by the server is based on the local model of the training end as an upstream to complete control and cooperation, and to realize the complete task.

[0056] Specifically, as shown in Figure 2 The method comprises the following steps S101-S106:

[0057] Step S101: The server registers with the smart contract and provides server registration information to initiate recruitment of the training end; the training end responds to the recruitment and registers with the smart contract to provide training end registration information.

[0058] Step S102: The training end performs forward propagation on the local model based on the local data sample to generate intermediate results and sends them to the service end smart contract bridge through the training end smart contract bridge for aggregation and then forwarding to the service end.

[0059] Step S103: The server performs forward propagation on the top model based on the aggregated intermediate results, calculates the loss and gradient information using the locally held labels, and updates the top model in reverse.

[0060] Step S104: The server distributes the loss and gradient information to the training end smart contract bridge through the service end smart contract bridge for each training end to perform backward propagation update on the corresponding local model until a preset termination condition is reached.

[0061] Step S105: The service end encrypts and divides the top model into continuous multiple sub-models and distributes them to multiple training ends, and provides initial test input; the output of the sub-model in each training end is uploaded to the smart contract and downloaded and run by the next training end in sequence to obtain the final test result and return it to the service end.

[0062] Step S106: The service end calculates the difference between the prediction results of each feature in the missing or introduced state in a certain number of rounds, averages the difference to obtain the influence score of the feature. The influence scores of the feature values contained in the local data of each training end are summed up to obtain the contribution index of the corresponding training end, the contribution index is used to distribute the bonus pool, and the pledge is disposed.

[0063] In step S101, one of the participants acts as a service end to initiate training, registers with the smart contract and provides service end registration information, and the other participants respond to the training and register with the smart contract as training ends and provide registration information. During the registration process, the service end and the training end are allocated corresponding IDs and addresses, and communication is performed during the subsequent training process.

[0064] In steps S102-S104, based on the network structure described above, the training phase can be divided into training end forward propagation phase, aggregation phase, service end forward propagation phase, service end backward propagation phase, gradient information distribution and client backward propagation phase. Specifically, in this process, all data interactions are based on service end smart contract bridge and training end smart contract bridge for data transmission, and do not need to be processed and forwarded through the smart contract. The communication between the bridges needs to be encrypted, such as HTTPS communication or other encrypted communication. In this process, the bridges of both parties need to record the intermediate information generated by the training end for traceability and persistence and the received intermediate result information for verification and persistence.

[0065] Specifically in step S102, the local data of the training end can be different in the feature domain. The local data is input into the local model to perform forward propagation to obtain intermediate results, which are sent to the service end smart contract bridge through the training end smart contract bridge and aggregated. This aggregation based on the cooperation form of the local model can be divided into summation or splicing.

[0066] In step S103, the service end performs forward propagation on the top model based on the aggregated intermediate results, calculates the loss and gradient information based on the locally held labels, and updates the top model through backward propagation. This specific process is consistent with general model training, and the loss calculation and gradient information calculation can be set based on the specific task scenario.

[0067] In step S104, the server distributes the loss and gradient information calculated based on the label to the training end intelligent contract bridge through the server intelligent contract bridge, and the local model is updated by the training end. The termination condition can be set as a specified number of iterations or the loss reaching a set value.

[0068] In steps S105 and S106, the contribution of each training end is measured based on the quantitative indicators. Specifically, since the features contained in the local data of each training end are different, the contribution to model training is also different. In this application, the server establishes a bonus pool and requires the training end to provide a margin, and distributes the bonus based on the contribution degree, which can encourage users with higher contribution and make each participant more inclined to provide better training assistance.

[0069] In order to measure the contribution difference of different training ends due to different feature spaces, the server uses the improved algorithm of Wang based on the Monte Carlo algorithm (hereinafter referred to as the improved algorithm), which controls the calculation of SHAP value to a constant level through a large number of random sampling methods, and the result of this simulation calculation method is similar to that of the Monte Carlo algorithm.

[0070] Specifically, in order to improve the calculation efficiency and prevent model weight leakage, the server divides the top model into continuous sub-models, which are distributed to each training end through the intelligent contract, and provides an initial test input, which contains all the data spaces of the training end. During processing, each training end takes the output of the previous training end sub-model as input to perform calculation, and finally obtains the result. For example, the top model is divided into two sub-models a and b, sub-model a is deployed on training end a, and sub-model b is deployed on training end b. The server not only provides the sub-models, but also provides an initial test input, which contains all the features in all the data of the training end. Training end a downloads sub-model a and the initial test input from the intelligent contract, calculates and uploads output data a; training end b downloads sub-model b and output data a from the intelligent contract, calculates and uploads output result b. The server downloads the output result b to calculate the contribution value SHAP.

[0071] In some embodiments, the influence score of each feature is obtained by calculating the difference between the prediction results of the feature in the absence or presence state in a certain number of rounds, including steps S201-S202:

[0072] Step S201: Construct two samples and The difference is only in the feature .

[0073] Step S202: The influence score of the fth feature is calculated as follows:

[0074] ;

[0075] wherein, the function m represents the prediction result, and N represents the set quantity.

[0076] For the impact score of the fth feature after N rounds of testing, the feature domain of each training end is different, and the impact score of the feature contained in each training end feature domain can be summed to obtain the contribution index of the training end.

[0077] The calculation formula of the contribution index of the training end is:

[0078] ;

[0079] wherein, represents the contribution index of the jth training end in the ith round, and F represents the number of features in the local data.

[0080] In some embodiments, the contribution index is modified,

[0081] ;

[0082] wherein, represents the modified contribution index, represents the accuracy value of the ith training end in the pre-test process, and M represents the number of training ends. The pre-test process can be performed before steps S105 and S106, that is, the local model trained by the training end is tested by the reserved test set, and the accuracy value is calculated.

[0083] In some embodiments, the bonus pool is distributed according to the contribution index, and the disposal of the pledge includes steps S301-S302:

[0084] Step S301: screening out unqualified training ends with a contribution index lower than a preset contribution threshold, and deducting the pledge of the unqualified training ends.

[0085] Step S302: calculating the contribution proportion of the training ends participating in the training except the unqualified training ends according to the contribution index, and distributing the bonus in the bonus pool to each training end participating in the training according to the contribution proportion; the calculation formula of the contribution proportion is:

[0086]

[0087] wherein, represents the contribution proportion of the ith client, M represents the number of training ends participating in the training, represents the modified contribution index.

[0088] ​The bonus calculation formula for the jth training end is:

[0089] ;

[0090] wherein Pool represents the prize pool amount.

[0091] In some embodiments, with reference to Figure 3 , the service end and the training end are each provided with an IPFS client to store the intermediate results generated in each round of training to the Interstellar File System, persist based on the blockchain, and persist the intermediate results saved in the Interstellar File System after each round of training, which includes integrity verification, data storage node verification, and storage retrieval reliability verification.

[0092] In some embodiments, the method further includes: in the process of registering the service end and the training end with the smart contract, the smart contract respectively issues asymmetric keys to the service end and the training end, broadcasts the public key in the asymmetric key, and the private key in the asymmetric key is held by the service end and the training end respectively, and in the process of transmitting the intermediate results, the loss, and the gradient information, the respective held asymmetric keys are used for encrypted communication.

[0093] In some embodiments, the method further includes: marking unqualified training ends and limiting their participation in subsequent training.

[0094] In another aspect, the present application also provides a decentralized heterogeneous neural network federated learning system, which includes a plurality of participants to jointly implement, one of the participants acts as a service end and the other plurality of participants act as training ends; the local data samples of the service end and the training ends are aligned; the service end only holds labels and is connected to the outside through a service end smart contract bridge, and the training end holds data and is connected to the outside through a training end smart contract bridge; the system executes the above-mentioned decentralized heterogeneous neural network federated learning method.

[0095] In another aspect, the present application also provides a computer readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method.

[0096] In another aspect, the present application also provides a computer program product including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method.

[0097] The present application will be described below in conjunction with a specific embodiment:

[0098] The federated learning system provided by the embodiment is applicable to a case where there is one active party (holding labels and training data) and multiple passive parties (only holding training data) in the system. In the starting stage, all parties in the system complete sample alignment, specifically, the sample space of each party is consistent, and the feature space is different. The federated learning is initiated by the active party, and an effective model is trained jointly by the passive parties.

[0099] In the vertical federated learning, the party node with both labels and features is referred to as an active party, and the party node with only features is referred to as a passive party.

[0100] The system includes multiple parties, each of which is composed of a training end, a C bridge and an IPFS client, or a service end, an S bridge and an IPFS client.

[0101] Training end: refers to a node that only has data, which can belong to two types of nodes. One is a passive party node, and the other is an active party node after removing the label column, and the two are not specially distinguished.

[0102] Service end: refers to a node that only has labels, that is, a node that only holds labels of an active party.

[0103] Training end smart contract bridge (referred to as C bridge): responsible for information transmission and smart contract interaction between nodes in the training end.

[0104] Service end smart contract bridge (referred to as S bridge): responsible for information transmission and smart contract interaction between nodes in the service end.

[0105] IPFS client: IPFS is a distributed file storage system, which is used to store a large amount of information in the exchange intermediate result.

[0106] Smart contract: the smart contract stores information of all participants, including a bonus pool stored by the active party. At the same time, the smart contract stores historical data of model training to ensure traceability of the training process.

[0107] As shown in Figure 1 The technology involves multiple parties. Any one of the parties includes an IPFS client, a training end (service end) and a C bridge (S bridge). Each federated client needs to establish a connection with the smart contract through a contract address. The features and labels of the original active party are divided into two training ports, one as a service end and the other as a training end. The service end only contains label information, and the training end only contains features.

[0108] Decentralized federated learning platform: complete decentralized training using off-chain training-on-chain persistence. Off-chain training-on-chain persistence specifically refers to the use of HTTPS (or other encrypted communication) communication to complete training, with the server leading and the training end cooperating, uploading persistent information to the smart contract and completing persistent verification at each round (every n rounds).

[0109] Training end: one end of vertical federated learning that only has features, whose purpose is to cooperate with the server end to complete model training and prediction. It shares information by uploading / downloading gradient information to the IPFS network.

[0110] Server end: one end of vertical federated learning that only has labels, whose work is to lead the training process and complete model training and prediction tasks with the cooperation of the training end. Similarly, it shares information by uploading / downloading gradient information to the IPFS network.

[0111] IPFS client: multiple IPFS clients form an IPFS network. Its main task is to complete the exchange of a large amount of data in federated learning training, and it is a decentralized file system. After uploading a file, a unique CID will be generated, and other clients can use the CID to download the corresponding file.

[0112] Smart contract: a customized smart contract that manages multiple states and various information in the system. For example, the current iteration round, node list, current state, training persistence information, and other related data reflecting the current state of the system. In addition, the smart contract not only records the current state, but also retains the history of past training to facilitate query and historical analysis.

[0113] The system execution process is as follows:

[0114] 1. Registration stage

[0115] The registration stage is divided into two parts, server registration and training end registration.

[0116] Server registration: the server registers with the smart contract, provides server registration information and bonus pool, and opens training end recruitment.

[0117] Training end registration: the training end registers with the smart contract, provides training end registration information and initial deposit. The deposit serves as a guarantee for the training end to actively participate in training, and the training end that normally participates in training will be refunded the deposit after training is completed.

[0118] 2. Persistence stage

[0119] For example Figure 3As shown, in the vertical federated training, the information interaction between the server and the training end is frequent, so it is not possible to directly collect the sub-model by the smart contract as in the horizontal federated learning. Therefore, the training method of off-chain training-on-chain persistence is proposed, and the training stage and the contribution calculation stage use encrypted communication to interact. The timing of persistence is performed before every n rounds, and the persistence process should be supervised to ensure that Sbridge and Cbridge correctly perform persistence. Therefore, after persistence, the persistence needs to be verified by Sbridge (Cbridge) to verify the persistence information of Cbridge (Sbridge), and the result of verifying the persistence determines whether a new round of training can be started. The larger n is, the faster the model training speed is. Therefore, the following process is introduced.

[0120] Persistence: Sbridge and Cbridge will upload the data generated during the training process to the smart contract at the end of each round (every n rounds), and the persistence process is a background process that does not affect the training speed of the main process.

[0121] Get persistence information: In the persistence information acquisition stage, Sbridge and Cbridge will poll the persistence information obtained from the smart contract.

[0122] Local verification: The obtained information is verified with the local information.

[0123] Submit verification result: The verification result is submitted to the smart contract, and the smart contract will collect all the verification results. When all the verification results are True, the next round is started, otherwise the training result of this round is invalid.

[0124] 3. Training stage

[0125] In the training of neural networks, the server has part of the neural network, usually referred to as the top model; each participant has part of the neural network, usually referred to as the local model. The training stage of the neural network is divided into training end forward propagation, aggregation stage, server forward propagation, server backward propagation, gradient downlink and client backward propagation stage.

[0126] Training end forward propagation: The training end first performs forward propagation, and sends the intermediate result matrix generated by forward propagation to Sbridge by Cbridge. In this stage, Sbridge will collect all the intermediate result matrices generated by the training end.

[0127] Aggregation stage: After the collection of the intermediate result matrix is completed, Sbridge will aggregate the intermediate result matrix in the form of summation or splicing. Finally, Sbridge sends the aggregated intermediate result matrix to the server.

[0128] Server forward propagation: The server performs forward propagation according to the aggregated intermediate result matrix and calculates the loss Loss and gradient information.

[0129] Server back propagation: the server updates the model according to the loss and gradient information.

[0130] Gradient distribution: the server sends the loss and gradient information to the S-bridge, the S-bridge distributes the gradient information to the C-bridge, and the C-bridge sends the gradient information to the corresponding training end.

[0131] Client back propagation: the client updates the model according to the loss and gradient information.

[0132] 5. Contribution calculation

[0133] In this system, the quantitative indicator SHAP is the first standard for measuring node contribution. The calculation of this value involves the allocation of benefits, so it is more fair to involve multiple parties.

[0134] ① Calculation method

[0135] Using Wang's improved algorithm based on Monte Carlo algorithm (hereinafter referred to as improved algorithm), the calculation of SHAP value is controlled in constant level through a large number of random sampling methods, and the calculation method of this simulation is similar to the result of Monte Carlo algorithm. In specific operation, the difference between the prediction results of the sample when the feature is missing and introduced is also calculated. For each sample and feature , it will be calculated for rounds. In each round, a random sample and a random feature subset are selected, F is the whole feature set, then two new samples are constructed, where:

[0136] ;

[0137] It can be seen that and only differ in . Since , , and .

[0138] The average of the prediction value difference of each round is obtained, and the approximate SHAP value is obtained:

[0139] ;

[0140] Where, is the SHAP value of feature f in sample x, and m is the model output function.

[0141] Under this calculation method, Wang proved the joint importance of different features, which can be summed to obtain a contribution index for a participant in federated learning.

[0142] ② Model segmentation

[0143] like Figure 4 As shown, the server uses an encrypted and segmented model to divide the different hidden layers of the top model, such as... Figure 3 The model is split into sub-models, which are then distributed to different training endpoints. For example, hidden layer-1 is assigned to training endpoint-1, hidden layer-2 to training endpoint-2, and hidden layer-3 to training endpoint-3. Each training endpoint then calculates the intermediate results of SHAP. The purpose of using the improved algorithm is to determine the difference in sample prediction results when features are missing or included. Therefore, the result of multi-party computation of the model split is consistent with the final result of individual computation.

[0144] ③ Calculation process

[0145] Under the above settings, such as Figure 5 As shown, the active party can divide the model into multiple parts and send them to the training end in a random order. The training end then autonomously calculates the output values ​​of its respective sub-model, which are ultimately verified by the server. This process requires multiple rounds, and the final average value is taken as the system's SHAP activation parameter.

[0146] 6. Bonus Distribution

[0147] The prize money distribution occurs upon completion of the entire training process. After each training round, the smart contract tallies the training contributions of all training endpoints. At the end of training, the prize money distribution percentage is calculated based on the contributions of each training endpoint in each round.

[0148] This embodiment proposes a training method for a decentralized vertical federated learning neural network model based on smart contracts. It also proposes a method for multi-party joint calculation of SHAP values ​​to achieve contribution calculation in the federated learning neural network model. The data of each round is submitted to the smart contract for data persistence and verification persistence, which reduces the frequency of smart contract calls and improves the training speed of decentralized vertical federated learning.

[0149] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0150] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the foregoing edge computing server deployment method. The computer readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0151] To sum up, the decentralized heterogeneous neural network federated learning method and system, the server initiates recruitment, registers through a smart contract and provides a bonus pool, the training end responds and provides a deposit; the server only holds labels, and the training end only holds data, the complete task front end executes the local model of the upstream task and is deployed on the training end, and the top model of the back end is deployed on the server, the training end performs forward propagation on the local model based on the local data to obtain intermediate results, sends the intermediate results to the server smart contract bridge through the training end smart contract bridge, aggregates the intermediate results by the server smart contract bridge, performs forward propagation on the top model by the server, calculates loss and gradient information by using the local labels, performs backward propagation to update the top model, sends the loss and the gradient information to the training end smart contract bridge through the server smart contract, and then performs backward propagation on the local model by the training end to update the local model. In the case of eliminating data islands, the model parameters do not need to be interacted in the training process, the model weight leakage can be effectively prevented, and the user privacy can be ensured.

[0152] Further, the server establishes a bonus pool, and the training end provides a deposit; the top model is divided into a plurality of sub-models, the sub-models are deployed to the training end for continuous collaborative calculation on the basis of providing initial test input, the influence score of the features contained in each data is obtained, the contribution degree index of each training end is calculated, the bonus in the bonus pool is allocated based on the contribution degree index, and the deposit of the training end that does not meet the requirements is deducted, so as to encourage each training end to provide better training service and optimize the training effect.

[0153] Those of ordinary skill in the art will appreciate that the various illustrative components, systems and methods described in connection with the embodiments disclosed herein can be implemented as hardware, software, or both. The particular implementation is dependent on the specific application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application. When implemented in hardware, for example, the hardware can comprise an electronic circuit, an Application Specific Integrated Circuit (ASIC), a suitable firmware, a plug-in, a functional card, etc. When implemented in software, the elements of the application are the program or code segments to perform a specific task. The program or code segments can be stored in a machine-readable medium, or transmitted by a carrier wave as data signals over a transmission medium or communication link.

[0154] It is to be understood that the application is not limited to the particular configurations and processes described herein and shown in the drawings, which can be varied in accordance with the particular needs of the application. For the sake of brevity, conventional techniques and methods related to making and using the application can not be described in detail herein. In the above embodiments, several specific steps are described and illustrated in order to provide a thorough understanding of the present application. However, the process of the present application can be practiced with less than all of the described and illustrated steps, or in a different order than that described and illustrated.

[0155] In the present application, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments, and / or in combination with or instead of features of other embodiments.

[0156] The above description is only preferred embodiments of the present application, and is not intended to limit the present application. The embodiments of the present application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A decentralized heterogeneous neural network federated learning method, characterized in that, The method is implemented jointly by multiple parties, including a server and multiple training terminals; the local data samples of the server and the training terminals are aligned; the server only holds labels and connects to the outside world through a server-side smart contract bridge, and the training terminals hold data and connect to the outside world through a training-side smart contract bridge; the method includes the following steps: The server registers with the smart contract, providing server registration information and a bonus pool to initiate recruitment for the training end; the training end responds to the recruitment by registering with the smart contract, providing training end registration information and a deposit. The training end performs forward propagation on the local model based on local data samples to generate intermediate results, which are then sent to the server-side smart contract bridge via the training end's smart contract bridge for aggregation and forwarding to the server. The server performs forward propagation on the top model based on the aggregated intermediate results, calculates the loss and gradient information using the locally held labels, and updates the top model in reverse. The top model is the top part of the complete task, and the local models of each training end are used to execute the upstream tasks and together with the top model constitute the complete task. The server distributes the loss and gradient information to the training server smart contract bridge through the server smart contract bridge, so that each training server can perform backpropagation to update the corresponding local model until a preset termination condition is reached. The server encrypts and divides the top model into multiple consecutive sub-models, which are then distributed to multiple training terminals, and provides initial test input. The output of each sub-model in each training terminal is uploaded to the smart contract and downloaded and run by the next training terminal in sequence. Finally, the test results are returned to the server. The server calculates the difference in prediction results for each feature in a set number of rounds under missing or introduced states, and averages the results to obtain the influence score of that feature. The server then sums the influence scores of the feature values ​​contained in the local data of each training end to obtain the contribution index of the corresponding training end. The server then distributes the bonus pool according to the contribution index and disposes of the pledged funds.

2. The decentralized heterogeneous neural network federated learning method according to claim 1, characterized in that, Both the server and the training end are equipped with an IPFS client to store the intermediate results generated in each round of training to the InterPlanetary File System (IPS) for persistence based on blockchain. After each set number of training rounds, the intermediate results stored in the IPS are persistently verified. The persistence verification includes integrity checks, verification of data storage nodes, and verification of storage retrieval reliability.

3. The decentralized heterogeneous neural network federated learning method according to claim 1, characterized in that, The method further includes: during the registration process between the server and the training end with the smart contract, the smart contract distributes asymmetric keys to the server and the training end respectively, broadcasts the public key in the asymmetric key, and the private key in the asymmetric key is held by the server and the training end respectively. During the transmission of the intermediate results, the loss and the gradient information, encrypted communication is performed using the asymmetric keys held by each party.

4. The decentralized heterogeneous neural network federated learning method according to claim 1, characterized in that, The influence score of a feature is obtained by averaging the differences in prediction results for each feature across a set number of rounds, considering both missing and introduced states. Construct two samples and The difference lies only in the features ; The f-th feature The formula for calculating the impact score is: ； Wherein, the function m represents the prediction result, and N represents the set quantity.

5. The decentralized heterogeneous neural network federated learning method according to claim 4, characterized in that, The formula for calculating the contribution index of the training end is: ； in, Let F represent the contribution index of the j-th training end in the i-th round, and let F represent the number of features in the local data; The contribution index is then revised. ； in, This represents the revised contribution index. Let M represent the accuracy value of the i-th training endpoint during the pre-testing process, and M represent the number of training endpoints.

6. The decentralized heterogeneous neural network federated learning method according to claim 5, characterized in that, The bonus pool is distributed according to the contribution index, and the disposal of the pledged funds includes: Unqualified training terminals whose contribution index is lower than the preset contribution threshold are selected, and the deposit of the unqualified training terminals is deducted. The contribution ratio of the training terminals participating in the training (excluding the unqualified training terminals) is calculated based on the contribution index, and the prize money in the prize pool is distributed to each training terminal participating in the training according to the contribution ratio; the formula for calculating the contribution ratio is: in, Let M represent the contribution ratio of the i-th client, and M represent the number of training clients participating in the training. This represents the revised contribution index; The formula for calculating the bonus allocated to the j-th training endpoint is: ； Here, Pool represents the prize pool amount.

7. The decentralized heterogeneous neural network federated learning method according to claim 6, characterized in that, The method further includes: marking the unqualified training endpoints and restricting their participation in subsequent training.

8. A decentralized heterogeneous neural network federated learning system, characterized in that, The system is implemented jointly by multiple participants, one of whom acts as a server and the other multiple participants as training terminals; the local data samples of the server and the training terminals are aligned; the server only holds labels and is connected to the outside world through a server smart contract bridge, and the training terminals hold data and are connected to the outside world through a training smart contract bridge; the system executes the decentralized heterogeneous neural network federated learning method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images