Prediction method and apparatus in federated learning

Abstract

The present disclosure relates to the technical field of federated learning, and provides a prediction method and device in federated learning. The method comprises: predicting, by using participant models of a plurality of participants, to-be-predicted data to obtain a first prediction result; updating model parameters of the participant models of the plurality of participants based on the first prediction result; re-predicting, by using the participant models of the plurality of participants after the model parameters are updated, to the to-be-predicted data to obtain a second prediction result; and calculating a third prediction result according to the second prediction results corresponding to the plurality of participants. By using the above technical means, the problems in the prior art that in federated learning, the prediction result is not related to the training of the model, and only federated learning models can be used for prediction are solved.

Description

Technical Field

[0001] This disclosure relates to the field of federated learning technology, and more particularly to a prediction method and apparatus in federated learning. Background Technology

[0002] In joint learning, a neural network model is typically trained using data from multiple participants, resulting in a participant model for each participant. Based on these participant models, a joint learning model is trained at the center, and then used for prediction. If the prediction accuracy of the joint learning model is low, retraining is required. It is evident that current technology cannot directly link prediction results to model training, and it can only use the joint learning model for prediction, failing to fully utilize the participant models of each participant.

[0003] In realizing the present invention, the inventors discovered at least the following technical problems in the related technologies: in joint learning, the prediction results are not linked to the training of the model, and prediction can only be performed using joint learning models. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a prediction method, apparatus, electronic device, and computer-readable storage medium in joint learning to solve the problems in the prior art, such as the lack of connection between prediction results and model training in joint learning, and the fact that only joint learning models can be used for prediction.

[0005] A first aspect of this disclosure provides a prediction method in joint learning, comprising: using a participant model of multiple participants to predict data to be predicted, obtaining a first prediction result; updating the model parameters of the participant models of multiple participants based on the first prediction result; using the participant models of multiple participants with updated model parameters to re-predict the data to be predicted, obtaining a second prediction result; and calculating a third prediction result based on the second prediction results corresponding to the multiple participants.

[0006] A second aspect of this disclosure provides a prediction apparatus in joint learning, comprising: a first prediction module configured to predict data to be predicted using participant models of multiple participants to obtain a first prediction result; an update module configured to update the model parameters of the participant models of the multiple participants based on the first prediction result; a second prediction module configured to re-predict the data to be predicted using the participant models of the multiple participants after updating the model parameters to obtain a second prediction result; and a calculation module configured to calculate a third prediction result based on the second prediction results corresponding to the multiple participants.

[0007] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0008] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The beneficial effects of this disclosed embodiment compared to the prior art are as follows: A first prediction result is obtained by using the participant models of multiple participants to predict the data to be predicted; the model parameters of the participant models of multiple participants are updated based on the first prediction result; the data to be predicted is re-predicted using the participant models of multiple participants with updated model parameters to obtain a second prediction result; and a third prediction result is calculated based on the second prediction results corresponding to the multiple participants. By employing the above technical means, the problems in the prior art, such as the lack of connection between prediction results and model training in joint learning, and the limitation to using only joint learning models for prediction, can be solved, thus providing a new prediction method in joint learning. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of a joint learning architecture according to an embodiment of this disclosure;

[0012] Figure 2 This is a flowchart illustrating a prediction method in joint learning provided by an embodiment of this disclosure;

[0013] Figure 3 This is a schematic diagram of the structure of a prediction device in joint learning provided in an embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0015] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0016] Federation learning refers to the comprehensive utilization of multiple AI (Artificial Intelligence) technologies, under the premise of ensuring data security and user privacy, to collaboratively explore the value of data and foster new intelligent business forms and models based on joint modeling. Federation learning has at least the following characteristics:

[0017] (1) Participating nodes control their own data in a weakly centralized joint training mode to ensure data privacy and security in the process of co-creating intelligence.

[0018] (2) In different application scenarios, various model aggregation optimization strategies are established by using screening and / or combination of AI algorithms and privacy-preserving computing to obtain high-level and high-quality models.

[0019] (3) Under the premise of ensuring data security and user privacy, based on multiple model aggregation optimization strategies, obtain methods to improve the performance of the federated learning engine. The performance methods can be improved by solving problems such as parallel computing architecture, information interaction under large-scale cross-domain networks, intelligent perception, and anomaly handling mechanisms.

[0020] (4) Obtain the needs of multiple users in various scenarios, determine the true contribution of each joint participant through a mutual trust mechanism, and allocate incentives accordingly.

[0021] Based on the above approach, an AI technology ecosystem based on collaborative learning can be established, fully leveraging the value of industry data and promoting the implementation of scenarios in vertical fields.

[0022] A prediction method and apparatus in joint learning according to embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0023] Figure 1 This is a schematic diagram of a joint learning architecture according to an embodiment of this disclosure. Figure 1 As shown, the architecture of joint learning may include a server (central node) 101 and participants 102, 103 and 104.

[0024] In the joint learning process, a basic model can be established through server 101, which then sends this model to participants 102, 103, and 104 with whom it has established a communication connection. Alternatively, any participant can establish the basic model and upload it to server 101, which then sends it to other participants with whom it has established a communication connection. Participants 102, 103, and 104 construct models based on the downloaded basic structure and model parameters, train the models using local data, obtain updated model parameters, and encrypt and upload these updated model parameters to server 101. Server 101 aggregates the model parameters sent by participants 102, 103, and 104 to obtain global model parameters, which are then transmitted back to participants 102, 103, and 104. Participants 102, 103, and 104 iterate on their respective models based on the received global model parameters until the models converge, thus achieving model training. During the collaborative learning process, the data uploaded by participants 102, 103, and 104 are model parameters. Local data is not uploaded to server 101, and all participants can share the final model parameters. Therefore, collaborative modeling can be achieved while ensuring data privacy. It should be noted that the number of participants is not limited to the three mentioned above, but can be set as needed. This embodiment of the disclosure does not impose any restrictions on this.

[0025] Figure 2 This is a flowchart illustrating a prediction method in joint learning provided in an embodiment of this disclosure. Figure 2 The prediction method in joint learning can be derived from Figure 1 The server executes this. For example... Figure 2 As shown, the prediction methods in this joint learning include:

[0026] S201, using a participant model involving multiple participants to predict the data to be predicted, and obtaining the first prediction result;

[0027] S202, Update the model parameters of the participant model for multiple participants based on the first prediction result;

[0028] S203, using the participant model of multiple participants after updating the model parameters, to re-predict the data to be predicted, and obtain the second prediction result;

[0029] S204, Calculate the third prediction result based on the second prediction results corresponding to multiple participants.

[0030] The execution entity of this disclosure embodiment can be a prediction system, which includes multiple participants and a training center. Joint learning training is initiated by the training center, and the multiple participants provide training datasets. Currently, joint learning does not link the prediction results with the model training. However, this disclosure embodiment can update the model parameters of the participant models of multiple participants based on the first prediction result, update the model parameters of the participant models of multiple participants, and then use the participant models of multiple participants to re-predict the data to be predicted to obtain a second prediction result. Therefore, this disclosure embodiment links the prediction results with the model training.

[0031] The embodiments disclosed herein can be applied to predicting natural gas consumption. In predicting natural gas consumption, by conducting two predictions and calculating a third prediction result based on the result of the second prediction, the accuracy of the predicted natural gas consumption can be improved.

[0032] The process involves using a participant model with multiple participants to predict the data to be predicted, resulting in a first prediction result for each participant; updating the model parameters of the participant model for each participant based on the first prediction result for each participant; re-predicting the data to be predicted using the participant model with updated model parameters for each participant, resulting in a second prediction result for each participant; and calculating a third prediction result based on the second prediction results for multiple participants, which can be obtained by weighted summation of the multiple second prediction results.

[0033] According to the technical solution provided in this disclosure, a first prediction result is obtained by using a participant model of multiple participants to predict the data to be predicted; the model parameters of the participant models of multiple participants are updated based on the first prediction result; the participant models of multiple participants with updated model parameters are used to re-predict the data to be predicted to obtain a second prediction result; and a third prediction result is calculated based on the second prediction results corresponding to the multiple participants. By employing the above technical means, the problems in the prior art, such as the lack of connection between prediction results and model training in joint learning, and the limitation to using only joint learning models for prediction, can be solved, thus providing a new prediction method in joint learning.

[0034] In step S202, the model parameters of the participant models of multiple participants are updated based on the first prediction result, including: calculating the prediction probability corresponding to the data to be predicted using the training center according to the first prediction result corresponding to the multiple participants; calculating the residual corresponding to the data to be predicted through the target participant model of the target participant according to the prediction probability, wherein the multiple participants include the target participant and the multiple participant models include the target participant model; and updating the model parameters of the participant models of multiple participants based on the residual.

[0035] In the prediction system, multiple participants make predictions on the data to be predicted, obtaining corresponding first prediction results. These first prediction results are then sent to the training center. Based on these first prediction results, the training center uses a joint learning model to calculate the prediction probability of the data to be predicted. The joint learning model then sends the prediction probability to the target participant. The target participant's target participant model calculates the residual corresponding to the data to be predicted based on the prediction probability, and the target participant sends the residual to the other participants. The model parameters of the participant models of the multiple participants, including the other participants and the target participant, are updated based on the residual.

[0036] Based on the first prediction results corresponding to multiple participants, after calculating the prediction probability corresponding to the data to be predicted using the training center, the method further includes: calculating the loss corresponding to the data to be predicted using the target participant model based on the prediction probability; updating the model parameters of the joint learning model in the training center based on the loss, wherein the joint learning model is related to the multiple participant models; and using the joint learning model with updated model parameters to predict the data to be predicted to obtain the fourth prediction result.

[0037] The predicted probability can also be used to calculate the loss corresponding to the data to be predicted. The loss is used to update the model parameters of the joint learning model in the training center. The model parameters of the joint learning model are aggregated from the model parameters of multiple participating models. The loss is related to the residual. Using the joint learning model with updated model parameters to predict the data to be predicted can also yield a prediction result. Therefore, this embodiment of the disclosure links the prediction result with the model training.

[0038] After using the joint learning model with updated model parameters to predict the data to be predicted and obtaining the fourth prediction result, the method further includes: calculating the residuals corresponding to the data to be predicted through the target participant model based on the prediction probability; updating the model parameters of multiple participant models based on the residuals; using the multiple participant models with updated model parameters to re-predict the data to be predicted and obtaining the second prediction result; calculating the third prediction result based on the second prediction results corresponding to multiple participants; and updating the third prediction result using the fourth prediction result.

[0039] In order to improve the accuracy of prediction of the data to be predicted, the embodiments of this disclosure use a fourth prediction result to update the third prediction result. This can be understood as adding the fourth prediction result and the third prediction result together and averaging them to obtain a new prediction result.

[0040] Before executing step S201, that is, before using the participant models of multiple participants to predict the data to be predicted and obtain the first prediction result, the method further includes: determining the common samples of the training datasets of multiple participants through sample alignment operation; training a neural network model based on the common samples and the training dataset of each participant to obtain the participant model of each participant; aggregating the model parameters of the participant models of multiple participants to obtain global model parameters; and updating the model parameters of the neural network model based on the global model parameters to obtain the joint learning model of the training center.

[0041] In joint learning, multiple parties are involved, each with their own data, which may contain identical samples. With increasing emphasis on data confidentiality, the data interaction among multiple parties in joint learning should also prioritize data security. To balance the quality and confidentiality of interactions among multiple parties, shared samples should be publicly available, while non-shared samples should be kept confidential. To achieve this objective, this embodiment first determines the common samples in the training datasets of multiple parties through a sample alignment operation before performing joint learning training.

[0042] The common samples of the training datasets of multiple participants are determined through sample alignment operations, including: processing the training datasets of multiple participants using a Bloom filter to obtain the processing results corresponding to multiple participants; traversing the processing results corresponding to any two participants among the multiple participants to determine the common samples of any two participants among the multiple participants, wherein the sample alignment operation includes traversal; traversing the common samples and the processing results corresponding to any one of the remaining participants to update the common samples, until the processing results corresponding to all participants have been traversed, and then the traversal stops.

[0043] In a joint learning process with multiple participants, the common samples of two participants are first determined. Then, the common samples of this common sample and the third participant are determined, and this newly determined common sample is used as the latest common sample. Each time, the common sample of any remaining participant is determined, and this newly determined common sample is used as the latest common sample. The latest common sample at the point of traversal is used as the common sample of all participants.

[0044] Bloom filters can retrieve whether an element is in a set. This disclosure proposes a sample alignment method for joint learning, based on the working principle of Bloom filters. This disclosure is the first to apply Bloom filters to joint learning. The sample alignment operation can be a traversal operation used to determine the common samples of multiple participants. Since each sample in the sample data of each participant can correspond to an identifier, traversing the samples or elements in the processing results corresponding to each participant can be done by traversing the identifier corresponding to a sample in the processing results corresponding to each participant, and determining whether the sample exists by checking if the identifier exists in the processing results corresponding to a participant. The Bloom filter was proposed by Burton Howard Bloom in 1970. It is actually composed of a long binary vector and a series of random mapping functions. Bloom filters can be used to retrieve whether an element is in a set.

[0045] In one optional embodiment, the process includes: multiple participants and any two parties in the training center interacting via a cross-language communication protocol, with the content of the interaction being processed using homomorphic encryption; the multiple participants include a target participant, the multiple participant models include a target participant model, the target participant has a private key and a public key, and the other participants have public keys.

[0046] Homomorphic encryption is a commonly used encryption technique in cloud computing, and will not be elaborated upon here. Because only the target participant possesses the private key, and because all interactive content is homomorphically encrypted, only the target participant can decrypt it (other participants encrypt using their public keys). Therefore, the other participants send their respective second prediction results (encrypted data) to the target participant. The target participant decrypts the second prediction results sent by the other participants, thus obtaining all the decrypted second prediction results. Based on all the decrypted second prediction results, the target participant calculates the third prediction result.

[0047] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0048] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0049] Figure 3 This is a schematic diagram of a prediction device in joint learning provided in an embodiment of this disclosure. Figure 3 As shown, the prediction device in this joint learning includes:

[0050] The first prediction module 301 is configured to use a participant model of multiple participants to predict the data to be predicted and obtain a first prediction result.

[0051] Update module 302 is configured to update the model parameters of the participant models of multiple participants based on the first prediction result;

[0052] The second prediction module 303 is configured to re-predict the data to be predicted using the participant model of multiple participants after updating the model parameters, and obtain the second prediction result.

[0053] The calculation module 304 is configured to calculate a third prediction result based on the second prediction results corresponding to multiple participants.

[0054] The execution entity of this disclosure embodiment can be a prediction system, which includes multiple participants and a training center. Joint learning training is initiated by the training center, and the multiple participants provide training datasets. Currently, joint learning does not link the prediction results with the model training. However, this disclosure embodiment can update the model parameters of the participant models of multiple participants based on the first prediction result, update the model parameters of the participant models of multiple participants, and then use the participant models of multiple participants to re-predict the data to be predicted to obtain a second prediction result. Therefore, this disclosure embodiment links the prediction results with the model training.

[0055] The process involves using a participant model with multiple participants to predict the data to be predicted, resulting in a first prediction result for each participant; updating the model parameters of the participant model for each participant based on the first prediction result for each participant; re-predicting the data to be predicted using the participant model with updated model parameters for each participant, resulting in a second prediction result for each participant; and calculating a third prediction result based on the second prediction results for multiple participants, which can be obtained by weighted summation of the multiple second prediction results.

[0056] According to the technical solution provided in this disclosure, a first prediction result is obtained by using a participant model of multiple participants to predict the data to be predicted; the model parameters of the participant models of multiple participants are updated based on the first prediction result; the participant models of multiple participants with updated model parameters are used to re-predict the data to be predicted to obtain a second prediction result; and a third prediction result is calculated based on the second prediction results corresponding to the multiple participants. By employing the above technical means, the problems in the prior art, such as the lack of connection between prediction results and model training in joint learning, and the limitation to using only joint learning models for prediction, can be solved, thus providing a new prediction method in joint learning.

[0057] Optionally, the update module 302 is further configured to calculate the prediction probability corresponding to the data to be predicted using the training center based on the first prediction results corresponding to multiple participants; calculate the residual corresponding to the data to be predicted using the target participant model of the target participant based on the prediction probability, wherein the multiple participants include the target participant and the multiple participant models include the target participant model; and update the model parameters of the participant models of the multiple participants based on the residual.

[0058] In the prediction system, multiple participants make predictions on the data to be predicted, obtaining corresponding first prediction results. These first prediction results are then sent to the training center. Based on these first prediction results, the training center uses a joint learning model to calculate the prediction probability of the data to be predicted. The joint learning model then sends the prediction probability to the target participant. The target participant's target participant model calculates the residual corresponding to the data to be predicted based on the prediction probability, and the target participant sends the residual to the other participants. The model parameters of the participant models of the multiple participants, including the other participants and the target participant, are updated based on the residual.

[0059] Optionally, the update module 302 is further configured to calculate the loss corresponding to the data to be predicted based on the predicted probability through the target participant model; update the model parameters of the joint learning model in the training center based on the loss, wherein the joint learning model is related to multiple participant models; and use the joint learning model with updated model parameters to predict the data to be predicted to obtain a fourth prediction result.

[0060] The predicted probability can also be used to calculate the loss corresponding to the data to be predicted. The loss is used to update the model parameters of the joint learning model in the training center. The model parameters of the joint learning model are aggregated from the model parameters of multiple participating models. The loss is related to the residual. Using the joint learning model with updated model parameters to predict the data to be predicted can also yield a prediction result. Therefore, this embodiment of the disclosure links the prediction result with the model training.

[0061] Optionally, the update module 302 is further configured to calculate the residuals corresponding to the data to be predicted using the target participant model based on the predicted probability; update the model parameters of multiple participant models based on the residuals; re-predict the data to be predicted using the multiple participant models with updated model parameters to obtain a second prediction result; calculate a third prediction result based on the second prediction result corresponding to the multiple participants; and update the third prediction result using the fourth prediction result.

[0062] In order to improve the accuracy of prediction of the data to be predicted, the embodiments of this disclosure use a fourth prediction result to update the third prediction result. This can be understood as adding the fourth prediction result and the third prediction result together and averaging them to obtain a new prediction result.

[0063] Optionally, the first prediction module 301 is further configured to determine common samples of the training datasets of multiple participants through sample alignment operations; train a neural network model based on the common samples and the training dataset of each participant to obtain a participant model for each participant; aggregate the model parameters of the participant models of multiple participants to obtain global model parameters; and update the model parameters of the neural network model based on the global model parameters to obtain a joint learning model of the training center.

[0064] In joint learning, multiple parties are involved, each with their own data, which may contain identical samples. With increasing emphasis on data confidentiality, the data interaction among multiple parties in joint learning should also prioritize data security. To balance the quality and confidentiality of interactions among multiple parties, shared samples should be publicly available, while non-shared samples should be kept confidential. To achieve this objective, this embodiment first determines the common samples in the training datasets of multiple parties through a sample alignment operation before performing joint learning training.

[0065] Optionally, the first prediction module 301 is further configured to process the training dataset of multiple participants using a Bloom filter to obtain the processing results corresponding to the multiple participants; traverse the processing results corresponding to any two participants among the multiple participants to determine the common sample of any two participants among the multiple participants, wherein the sample alignment operation includes traversal; traverse the common sample and the processing results corresponding to any one of the remaining participants to update the common sample, until the processing results corresponding to all participants have been traversed, and then stop traversing.

[0066] In a joint learning process with multiple participants, the common samples of two participants are first determined. Then, the common samples of this common sample and the third participant are determined, and this newly determined common sample is used as the latest common sample. Each time, the common sample of any remaining participant is determined, and this newly determined common sample is used as the latest common sample. The latest common sample at the point of traversal is used as the common sample of all participants.

[0067] Bloom filters can retrieve whether an element is in a set. This disclosure proposes a sample alignment method for joint learning, based on the working principle of Bloom filters. This disclosure is the first to apply Bloom filters to joint learning. The sample alignment operation can be a traversal operation used to determine the common samples of multiple participants. Since each sample in the sample data of each participant can correspond to an identifier, traversing the samples or elements in the processing results corresponding to each participant can be done by traversing the identifier corresponding to a sample in the processing results corresponding to each participant, and determining whether the sample exists by checking if the identifier exists in the processing results corresponding to a participant. The Bloom filter was proposed by Burton Howard Bloom in 1970. It is actually composed of a long binary vector and a series of random mapping functions. Bloom filters can be used to retrieve whether an element is in a set.

[0068] Multiple participants and any two parties in the training center interact through a cross-language communication protocol, and the content of the interaction is processed through homomorphic encryption; the multiple participants include the target participant, the multiple participant model includes the target participant model, the target participant has a private key and a public key, and the other participants have public keys.

[0069] Homomorphic encryption is a commonly used encryption technique in cloud computing, and will not be elaborated upon here. Because only the target participant possesses the private key, and because all interactive content is homomorphically encrypted, only the target participant can decrypt it (other participants encrypt using their public keys). Therefore, the other participants send their respective second prediction results (encrypted data) to the target participant. The target participant decrypts the second prediction results sent by the other participants, thus obtaining all the decrypted second prediction results. Based on all the decrypted second prediction results, the target participant calculates the third prediction result.

[0070] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0071] Figure 4 This is a schematic diagram of the electronic device 4 provided in an embodiment of this disclosure. Figure 4As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0072] For example, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to perform the present disclosure. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in electronic device 4.

[0073] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0074] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0075] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 402 can include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0077] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0078] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0079] In the embodiments provided in this disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0081] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0082] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0083] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A prediction method in joint learning, characterized in that, include: A participant model involving multiple participants is used to predict the data to be predicted, and a first prediction result is obtained. The model parameters of the participant models of the multiple participants are updated based on the first prediction result; The data to be predicted is re-predicted using the participant models of multiple participants after the model parameters have been updated, to obtain a second prediction result; Calculate the third prediction result based on the second prediction results corresponding to the multiple participating parties; The step of updating the model parameters of the participant models of multiple participants based on the first prediction result includes: Based on the first prediction results corresponding to the multiple participants, the prediction probability corresponding to the data to be predicted is calculated using the training center. Based on the predicted probability, the residual corresponding to the data to be predicted is calculated through the target participant model of the target participant, wherein the plurality of participants include the target participant, and the plurality of participant models include the target participant model; The model parameters of the participant models of multiple participants are updated based on the residuals; After calculating the prediction probability corresponding to the data to be predicted using a training center based on the first prediction results corresponding to multiple participants, the method further includes: Based on the predicted probability, the loss corresponding to the data to be predicted is calculated using the target participant model; The model parameters of the joint learning model in the training center are updated according to the loss, wherein the joint learning model is associated with multiple participating party models; The joint learning model with updated model parameters is used to predict the data to be predicted, resulting in a fourth prediction result. After using the joint learning model with updated model parameters to predict the data to be predicted and obtaining a fourth prediction result, the method further includes: Based on the predicted probability, the residual corresponding to the data to be predicted is calculated using the target participant model; The model parameters of multiple participating party models are updated based on the residuals; The second prediction result is obtained by re-predicting the data to be predicted using multiple participant models with updated model parameters. The third prediction result is calculated based on the second prediction results corresponding to the multiple participating parties; The third prediction result is updated using the fourth prediction result; Any two of the participating parties and the training center can interact through a cross-language communication protocol, and the content of the interaction is processed through homomorphic encryption. The plurality of participants includes a target participant, and the plurality of participant models includes a target participant model. The target participant has a private key and a public key, and the other participants have the public key.

2. The method according to claim 1, characterized in that, Before obtaining the first prediction result by using a participant model with multiple participants to predict the data to be predicted, the method further includes: The common samples of the training datasets of the multiple participants are determined through a sample alignment operation; Train a neural network model based on the public samples and the training dataset of each participant to obtain a participant model for each participant; The model parameters of the participant models of multiple participants are aggregated to obtain the global model parameters; The model parameters of the neural network model are updated based on the global model parameters to obtain the joint learning model of the training center.

3. The method according to claim 2, characterized in that, The step of determining common samples of the training datasets of the multiple participants through sample alignment operations includes: The training datasets of the multiple participants are processed using a Bloom filter to obtain the processing results corresponding to the multiple participants; The processing results corresponding to any two of the multiple participants are traversed to determine the common sample of any two of the multiple participants, wherein the sample alignment operation includes traversal; The process involves iterating through the common sample and the processing results corresponding to any of the remaining participants to update the common sample, until all the processing results corresponding to the participants have been traversed, at which point the traversal stops.

4. A prediction device in joint learning, characterized in that, The prediction device employs the method according to any one of claims 1-3, and the prediction device comprises: The first prediction module is configured to use a participant model of multiple participants to predict the data to be predicted and obtain the first prediction result. The update module is configured to update the model parameters of the participant models of the multiple participants based on the first prediction result; The second prediction module is configured to re-predict the data to be predicted using the participant models of multiple participants after updating the model parameters, and obtain a second prediction result. The calculation module is configured to calculate a third prediction result based on the second prediction results corresponding to the multiple participants.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3.

Images