Federal learning-based model training method

By using the obfuscation factor data set and obfuscation commitment generated by the coordinator in federated learning for data encryption, the ciphertext expansion problem is solved, the model training efficiency and flexibility is improved, and multi-participant federated learning modeling is realized.

CN118944926APending Publication Date: 2024-11-12WEBANK (CHINA)
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410996351.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The homomorphic encryption algorithm used in the existing federated learning method leads to the problem of ciphertext expansion, increasing the amount of data operations and limiting the encryption device, and reducing the efficiency of model training.

Method used

Data encryption is performed through the obfuscation factor data set and obfuscation commitment generated by the coordinator, which reduces the amount of encrypted ciphertext data, improves model training efficiency, and lifts the limit on the number of participants by federated learning modeling.

Benefits of technology

It improves the efficiency and flexibility of federated learning model training, allows multiple participants to participate in federated learning modeling, avoids the problem of ciphertext expansion, and enhances data privacy protection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118944926A_ABST
    Figure CN118944926A_ABST
Patent Text Reader

Abstract

The invention discloses a federated learning-based model training method. The method comprises the steps of obtaining a confusion factor data set with a preset data length and a confusion commitment which are sent by a coordinator and correspond to a current participant; wherein the confusion factor data set and the confusion commitment are generated by the coordinator based on sample data provided by a plurality of participants and the number of the participants; performing data encryption processing according to the confusion factor data set and the confusion commitment to obtain encrypted data of the current participant; performing model calculation based on the encrypted data of the current participant to obtain a calculation result of the current participant; and sending the calculation result of the current participant to the coordinator to obtain a summarized calculation result fed back by the coordinator based on the calculation results of the plurality of participants, and adjusting the model parameters according to the summarized calculation result. According to the technical scheme, the data volume of the encrypted ciphertext data is reduced, the data volume involved in the subsequent model training process can be further reduced, and therefore the model training efficiency is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of federated learning technology, and specifically relates to a model training method based on federated learning. Background Art

[0002] Federated Learning is a distributed machine learning method that aims to solve the problems of data privacy and security. Federated learning aggregates the information of model parameter updates by training on local devices, thereby achieving model training while protecting data privacy. Existing solutions generally use semi-homomorphic or fully homomorphic encryption algorithms based on public key cryptography to build logistic regression protocols. The scheme is constructed using the traditional public key cryptography homomorphic encryption algorithm, which has a simple construction pattern and simple implementation. However, the homomorphic encryption algorithm itself is a complex operation, which cannot be supported by some light clients and will cause ciphertext expansion problems. The original data type may be 2 bytes, and the data encrypted by the homomorphic algorithm is 512 bytes. The entire data expansion will become 256 times, which will increase the amount of data calculation and the restrictions on encryption devices, making the federated learning modeling less efficient and slower. Summary of the invention

[0003] The purpose of this application is to provide a model training method based on federated learning to reduce ciphertext expansion in the federated learning process and improve model training efficiency.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a model training method based on federated learning is provided, including:

[0006] Obtaining a confusion factor data set and a confusion commitment of a preset data length corresponding to the current participant sent by the coordinator; wherein the confusion factor data set and the confusion commitment are generated by the coordinator based on sample data and the number of participants provided by multiple participants; the confusion factor data set includes multiple confusion factors of preset data lengths, the number of the confusion factors is the same as the number of sample data provided by multiple participants, and the confusion commitment corresponding to the current participant is generated according to the confusion factor corresponding to the sample data of the current participant;

[0007] Performing data encryption processing according to the obfuscation factor data group and the obfuscation commitment to obtain encrypted data of the current participant;

[0008] Performing model calculation based on the encrypted data of the current participant to obtain a calculation result of the current participant;

[0009] The calculation result of the current participant is sent to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and the model parameters are adjusted according to the summary calculation result.

[0010] According to one aspect of an embodiment of the present application, a model training device based on federated learning is provided, including:

[0011] A confusion data acquisition module, used to acquire a confusion factor data group and a confusion commitment of a preset data length corresponding to the current participant sent by the coordinator; wherein the confusion factor data group and the confusion commitment are generated by the coordinator based on sample data and the number of participants provided by multiple participants; the confusion factor data group includes multiple confusion factors of preset data lengths, the number of the confusion factors is the same as the number of sample data provided by multiple participants, and the confusion commitment corresponding to the current participant is generated according to the confusion factor corresponding to the sample data of the current participant;

[0012] An encryption processing module, used for performing data encryption processing according to the obfuscation factor data group and the obfuscation commitment to obtain the encrypted data of the current participant;

[0013] A model calculation module, used to perform model calculation based on the encrypted data of the current participant to obtain the calculation result of the current participant;

[0014] The parameter adjustment module is used to send the calculation result of the current participant to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and adjust the model parameters according to the summary calculation result.

[0015] In one embodiment of the present application, the number of the confusion factor data groups is the same as the number of the participants; the confusion factor data group includes multiple randomly generated confusion factors of preset data lengths, and one confusion factor corresponds to one sample data; the confusion commitment corresponding to the current participant is the sum of the multiple confusion factors corresponding to the sample data of the current participant.

[0016] In one embodiment of the present application, the encrypted data of the current participant includes multiple ciphertext data fragments; the encryption processing module is specifically used to:

[0017] Traversing the confusion factors in the confusion factor data group, if the sample data corresponding to the current confusion factor does not belong to the current participant, using the current confusion factor as the data ciphertext fragment of the sample data at the current participant;

[0018] If the sample data corresponding to the current confusion factor belongs to the current participant, the sample data is subtracted from the confusion commitment and then added with the current confusion factor as the ciphertext data fragment of the sample data at the current participant.

[0019] In one embodiment of the present application, the model calculation module includes:

[0020] A first calculation unit is used to calculate according to the encrypted data of the current participant and the model parameters to obtain a first secret parameter shard; wherein the model parameters have initial values; and the first secret parameter shard represents the predicted value of the model for the sample data;

[0021] A second calculation unit is used to perform multiple ciphertext multiplication operations on the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter slice; wherein the second secret parameter slice represents a characteristic value after the model processes the sample data;

[0022] A third calculation unit is used to perform a ciphertext multiplication operation on the encrypted data of the current participant and the second secret parameter slice to obtain a third secret parameter slice; wherein the third secret parameter slice represents a model gradient;

[0023] a fourth calculation unit, configured to calculate a fourth secret parameter slice according to the first secret parameter slice; wherein the second secret parameter slice represents a model loss;

[0024] A result generating unit is used to use the first secret parameter slice, the second secret parameter slice, the third secret parameter slice and the fourth secret parameter slice as the calculation result of the current participant.

[0025] In one embodiment of the present application, the sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the training data; the model parameters include the first model parameters and the second model parameters; the first calculation unit is specifically used to:

[0026] Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant;

[0027] The product of the data feature sum slice of the current participant and the first model parameter is added to the second model parameter to obtain the first secret parameter slice.

[0028] In one embodiment of the present application, the sample data provided by the multiple participants include label data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the label data; and the second computing unit is specifically used to:

[0029] Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter stage one slice;

[0030] Multiplying the second secret parameter stage one slice by the first set coefficient to obtain the second secret parameter stage two slice;

[0031] Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the second secret parameter stage two slice to obtain the second secret parameter stage three slice;

[0032] The second secret parameter stage three slice is subtracted from the product of the data ciphertext slice corresponding to the tag data and the second set coefficient to obtain the second secret parameter slice.

[0033] In one embodiment of the present application, the second computing unit is specifically configured to:

[0034] Obtaining a first multiplication confusion factor slice, a second multiplication confusion factor slice, and a third multiplication confusion factor slice corresponding to the current participant; wherein each multiplication confusion factor slice is generated by slicing the corresponding multiplication confusion factor according to the number of participants, the first multiplication confusion factor and the second multiplication confusion factor are data of the preset data length randomly generated by the coordinator, and the third multiplication confusion factor is the product of the first multiplication confusion factor and the second multiplication confusion factor;

[0035] Obtaining a first intermediate shard of the current participant according to a difference between the first secret parameter shard and the first multiplication confusion factor shard, and obtaining a second intermediate shard of the current participant according to a difference between a data ciphertext shard corresponding to the tag data and the second multiplication confusion factor shard;

[0036] Sending the first intermediate shard and the second intermediate shard to the coordinator to obtain a first summary shard and a second summary shard fed back by the coordinator, wherein the first summary shard is obtained by the coordinator summarizing the first intermediate shards of multiple participants, and the second summary shard is obtained by the coordinator summarizing the second intermediate shards of multiple participants;

[0037] The second secret parameter phase one slice is generated according to the first multiplication confusion factor slice, the second multiplication confusion factor slice, the third multiplication confusion factor slice, the first summary slice, and the second summary slice.

[0038] In one embodiment of the present application, the second computing unit is specifically configured to:

[0039] Multiplying the first multiplication confusion factor slice and the first summary slice to obtain a first product;

[0040] Multiplying the second multiplication confusion factor slice and the second summary slice to obtain a first product;

[0041] multiplying the first summary slice and the second summary slice to obtain a third product;

[0042] The sum of the first product, the second product, the third product and the third multiplication confusion factor shard is used as the second secret parameter stage one shard.

[0043] In one embodiment of the present application, the sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the training data; and the third computing unit is specifically used to:

[0044] Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant;

[0045] A ciphertext multiplication operation is performed on the data feature sum slice and the second secret parameter slice to obtain the third secret parameter slice.

[0046] In one embodiment of the present application, the fourth computing unit is specifically configured to:

[0047] Performing a ciphertext multiplication operation on the first secret parameter slice to obtain a fourth secret parameter stage one slice;

[0048] Multiplying the fourth secret parameter stage one slice by a third set coefficient to obtain a fourth secret parameter stage two slice;

[0049] The fourth secret parameter stage two slice is subtracted from the product of the second secret parameter stage one slice and the fourth setting coefficient to obtain the fourth secret parameter slice.

[0050] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the model training method based on federated learning in the above technical solution is implemented.

[0051] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor executes the executable instructions so that the electronic device performs a model training method based on federated learning as in the above technical solution.

[0052] According to one aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the model training method based on federated learning in the above technical solution.

[0053] In the technical solution provided in the embodiment of the present application, the current participant performs data encryption processing based on the confusion factor data group and the confusion commitment to obtain the encrypted data of the current participant; and the confusion factor data group and the confusion commitment are data of preset data length generated by the coordinator based on the sample data provided by multiple participants and the number of participants. The data encryption process does not need to introduce other data like the homomorphic algorithm, and only needs to be calculated according to the confusion factor data group and the confusion commitment, thereby reducing the data volume of the encrypted ciphertext data, and then can reduce the data volume involved in the subsequent model training process, thereby improving the model training efficiency; at the same time, after the current participant performs model calculation based on the encrypted data, the calculation result is sent to the coordinator, so that the coordinator summarizes the calculation results of multiple participants. Finally, the current participant adjusts the model parameters according to the summarized calculation results fed back by the coordinator to realize model training. The coordinator plays a data scheduling role in the federated learning modeling process of multiple participants, thereby removing the restriction on the number of participants in federated learning modeling. More than two participants can also realize federated learning modeling through this solution, which improves the flexibility of federated learning modeling.

[0054] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0057] Figure 2 A flowchart of a model training method based on federated learning provided by an embodiment of the present application is schematically shown.

[0058] Figure 3A flowchart of a model training method based on federated learning provided by an embodiment of the present application is schematically shown.

[0059] Figure 4 A schematic diagram of a model training method based on federated learning provided in one embodiment of the present application is schematically shown.

[0060] Figure 5 The structural block diagram of the model training device based on federated learning provided in an embodiment of the present application is schematically shown.

[0061] Figure 6 The structure block diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0062] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0063] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0064] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0065] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0066] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0067] It is understandable that in the specific implementation of this application, related data such as customer information (such as transaction information, reconciliation data) is involved. When the above embodiments of this application are applied to specific products or technologies, customer permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0068] Figure 1 The exemplary system architecture block diagram applying the technical solution of the present application is schematically shown.

[0069] like Figure 1 As shown, the system architecture 100 may include a participant 110 and a coordinator 120. The participant 110 and the coordinator 120 may be terminal devices or servers in any form, for example, the terminal devices may include but are not limited to smart phones, tablet computers, laptops, intelligent voice interaction devices, smart home appliances, vehicle terminals, etc. The server 130 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The participant 110 and the coordinator 120 may be connected via a network, and the network 0 may be a communication medium of various connection types that can provide a communication link between the participant 110 and the coordinator 120, for example, a wired communication link or a wireless communication link.

[0070] In this embodiment, the coordinator 120 is used to perform data preprocessing, network routing and data aggregation. Data preprocessing refers to generating a confusion factor data set and a confusion commitment based on sample data and the number of participants provided by multiple participants 110, and sending the confusion factor data set and the confusion commitment to each participant 110. Network routing refers to determining which participant 110 the data is specifically sent to. Data aggregation includes aggregating the calculation results of each participant 110. According to implementation needs, the system architecture in the embodiment of the present application can have any number of participants 110.

[0071] In the SaaS (Software as a Service) scenario, there is usually a central server that provides services and multiple users, and these users may have their own data. However, for data privacy and security considerations, users usually do not want to transfer their data to the central server for processing or analysis. In this case, federated learning can be used to jointly calculate the data of multiple users. For example, in the loan risk control scenario, the data of the same user on different platforms often have a gain effect on the logistic regression model (the logistic regression model is a type of federated learning model). For example, there is a user list user_list = [u1, u2, u3, .., un], such as [Zhang San, Li Si, ...], in which each user has applied for a loan from the loan bank A, and bank A has the user's ID number id, whether it is overdue y and other data; at the same time, the users in the user list user_list have ID number id, consumption record x1 and other data on the e-commerce platform B; the users in the user list user_list have ID number id, payment record x2, call record and other data on the operator C; the users in the user list user_list have ID number id, tax record x3 and other data on the tax bureau D. The existing platform party S (such as the Government Services and Data Management Bureau, referred to as the Government Services and Data Management Bureau) matches the above-mentioned bank A, e-commerce platform B, operator C and tax bureau D to use their respective data for federated learning modeling to obtain a trained federated learning model, which can be used to predict the probability of loan delinquency for new users. The above-mentioned bank A, e-commerce platform B, operator C and tax bureau D are the various participants in the technical solution of this application, and the platform party S is equivalent to the coordinator in the technical solution of this application. The overdue y data provided by bank A, the consumption record x1 data provided by e-commerce platform B, the payment record x2 data provided by operator C and the tax record x3 data provided by tax bureau D belong to the sample data of each participant, among which the overdue y data provided by bank A belongs to the label data in the sample data, and the consumption record x1 data provided by e-commerce platform B, the payment record x2 data provided by operator C and the tax record x3 data provided by tax bureau D belong to the training data in the sample data.

[0072] The model training method based on federated learning provided in this application is described in detail below in conjunction with specific implementation methods.

[0073] Figure 2 The flowchart of a model training method based on federated learning provided by an embodiment of the present application is schematically shown, and the method can be implemented by participants in the federated learning calculation. Figure 2 As shown, the model training method based on federated learning provided in this embodiment includes steps 210 to 240, which are specifically as follows:

[0074] Step 210, obtaining a confusion factor data group and a confusion commitment of a preset data length corresponding to the current participant sent by the coordinator; wherein the confusion factor data group and the confusion commitment are generated by the coordinator based on sample data and the number of participants provided by multiple participants; the confusion factor data group includes multiple confusion factors of preset data lengths, the number of confusion factors is the same as the number of sample data provided by the multiple participants, and the confusion commitment corresponding to the current participant is generated based on the confusion factor corresponding to the sample data of the current participant.

[0075] Specifically, before the federated learning model calculation is performed, the coordinator performs data preprocessing, that is, pre-generates the confusion factor data set and confusion commitment required by each participant. Each participant registers the number of sample data provided by it to the coordinator according to the current federated learning model calculation task. For example, Bank A (hereinafter also recorded as the first participant or participant 1) registers 1 piece of data (whether it is overdue y data) with the coordinator, e-commerce platform B (hereinafter also recorded as the second participant or participant 2) registers 1 piece of data (consumption record x1 data) with the coordinator, operator C (hereinafter also recorded as the third participant or participant 3) registers 1 piece of data (payment record x2 data) with the coordinator, and tax bureau D (hereinafter also recorded as the fourth participant or participant 4) registers 1 piece of data (tax payment record x3 data) with the coordinator. It can be understood that the number of data registered by each participant can be the same or different.

[0076] The coordinator generates a confusion factor data set and a confusion commitment based on the sample data and the number of participants provided by multiple participants. When registering the number of sample data, the participants can inform the coordinator of the corresponding data identifier so that the coordinator can identify the sample data based on the data identifier. For example, Bank A identifies the overdue y data as "data y" and registers it with the coordinator. When the coordinator identifies the data identifier "data y", it knows that the data is provided by Bank A. In this way, the participants only need to provide the data identifier without providing specific data to the coordinator, which effectively guarantees the privacy of the participants' data. For each sample data, the coordinator generates a confusion factor corresponding to each participant. Then the coordinator needs to generate a number of confusion factors corresponding to the number of participants for one sample data. Assuming that the sample data is recorded as data (according to the above example, the values ​​of data include y, x1, x2 and x3), the confusion factor corresponding to the i-th participant is recorded as o_data_i. For example, there are 4 participants in the above example. For data y, 4 confusion factors are generated: o_y_1, o_y_2, o_y_3, o_y_4. When generating the confusion factor, the coordinator generates a random number as the confusion factor based on the preset data length. For example, the preset data length is 256 bits, which means that the confusion factor is a binary data with a 256-bit number.

[0077] In one embodiment of the present application, all confusion factors corresponding to the same participant are grouped together to form a confusion factor data group, so each participant corresponds to a confusion factor data group, and the number of confusion factor data groups is the same as the number of participants. For example, for participant 1, its corresponding confusion factor data group includes o_y_1, o_x1_1, o_x2_1 and o_x3_1. At the same time, all confusion factors corresponding to the same sample data are summed to obtain the confusion commitment corresponding to the sample data, for example, the confusion commitment corresponding to data y is o_data_sum=o_y_1+o_y_2+o_y_3+o_y_4. Finally, the coordinator sends each confusion factor to the participant corresponding to the confusion factor, and sends the confusion commitment of the sample data to the participant who provided the sample data. Thus, the confusion factors received by the participant constitute a confusion factor data group, which includes the confusion factor generated for each sample data for the participant, and the confusion commitment corresponding to the participant is the confusion commitment corresponding to the sample data provided by the participant. Exemplarily, participant 1 obtains the obfuscation factor data set [o_y_1, o_x1_1, o_x2_1, o_x3_1] and the obfuscation commitment o_data_sum=o_y_1+o_y_2+o_y_3+o_y_4.

[0078] Step 220: Perform data encryption processing according to the obfuscation factor data set and the obfuscation commitment to obtain the encrypted data of the current participant.

[0079] Specifically, the current participant is any one of the multiple participants. After obtaining the obfuscation factor data set and the obfuscation commitment from the coordinator, the current participant uses the two obfuscation data to encrypt local data to generate encrypted data.

[0080] In one embodiment of the present application, the encrypted data is also called a data ciphertext slice. The encrypted data generated by the current participant includes not only the data ciphertext slice corresponding to the sample data provided by itself, but also the data ciphertext slice formed for the sample data of other participants. The specific encryption process includes: traversing the confusion factors in the confusion factor data group. If the sample data corresponding to the current confusion factor does not belong to the current participant, the current confusion factor is used as the data ciphertext slice of the sample data of the current participant; if the sample data corresponding to the current confusion factor belongs to the current participant, the sample data is subtracted from the confusion commitment and then the current confusion factor is added as the data ciphertext slice of the sample data of the current participant.

[0081] Specifically, the current participant traverses the confusion factors in the confusion factor data group, and detects whether the sample data corresponding to the current confusion factor traversed belongs to the current participant. If not, the current confusion factor is directly used as the corresponding data ciphertext shard; if it does, the sample data is encrypted using the current confusion factor and the confusion commitment to obtain the corresponding data ciphertext shard, which is the data obtained by subtracting the confusion commitment from the sample data and adding the current confusion factor. Exemplarily, participant 1 has a confusion factor data set [o_y_1, o_x1_1, o_x2_1, o_x3_1] and a confusion commitment o_data_sum=o_y_1+o_y_2+o_y_3+o_y_4. When traversing to the confusion factor o_y_1, the sample data y corresponding to the confusion factor is the sample data provided by participant 1, so the corresponding data ciphertext slice share_y=y-o_data_sum+o_y_1; when traversing to the confusion factor o_x1_1, the sample data x1 corresponding to the confusion factor is not the sample data provided by participant 1, so the corresponding data ciphertext slice share_x1=o_x1_1. Similarly, the data ciphertext slice share_x2=o_x2_1 corresponding to the confusion factor o_x2_1, and the data ciphertext slice share_x3=o_x3_1 corresponding to the confusion factor o_x3_1. As a result, participant 1 obtains the encrypted data [share_y, share_x1, share_x2, share_x3].

[0082] Step 230: Perform model calculation based on the encrypted data of the current participant to obtain the calculation result of the current participant.

[0083] Specifically, after obtaining the encrypted data, the current participant performs model calculation locally based on the encrypted data, and the calculation results obtained include the model's predicted value for the sample data, the model's feature value after processing the sample data, the model gradient, and the model loss. When a participant does not obtain the sample data of other participants, the participant can use the data ciphertext fragments corresponding to the sample data to replace the sample data of other participants for model calculation, so that each participant can perform model calculation without exposing its own sample data to other participants, thereby effectively ensuring the security of each participant's data during the model training process.

[0084] Step 240: Send the calculation result of the current participant to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and adjust the model parameters according to the summary calculation result.

[0085] Specifically, after the current participant obtains the calculation result, it sends the calculation result to the coordinator. The coordinator receives the calculation results of multiple participants, performs summary calculations, obtains the summary calculation results, and feeds back to each participant. Each participant adjusts the model parameters according to the summary calculation results, and then uses the sample data for iterative calculations again until the convergence conditions are reached to obtain a trained federated learning model. Optionally, the gradient descent algorithm can be used to optimize the model, that is, by calculating the gradient of the loss function with respect to the model parameters, and updating the model parameters according to the direction of the gradient to gradually approach the optimal solution. By continuously iteratively updating the model parameters, the model can be made to better fit the data. The convergence condition can be that the number of iterations reaches the set number, or the model loss meets the preset conditions.

[0086] In the technical solution provided in the embodiment of the present application, the current participant performs data encryption processing based on the confusion factor data group and the confusion commitment to obtain the encrypted data of the current participant; and the confusion factor data group and the confusion commitment are data of preset data length generated by the coordinator based on the sample data provided by multiple participants and the number of participants. The data encryption process does not need to introduce other data like the homomorphic algorithm, and only needs to be calculated according to the confusion factor data group and the confusion commitment, thereby reducing the data volume of the encrypted ciphertext data, and then can reduce the data volume involved in the subsequent model training process, thereby improving the model training efficiency; at the same time, after the current participant performs model calculation based on the encrypted data, the calculation result is sent to the coordinator, so that the coordinator summarizes the calculation results of multiple participants. Finally, the current participant adjusts the model parameters according to the summarized calculation results fed back by the coordinator to realize model training. The coordinator plays a data scheduling role in the federated learning modeling process of multiple participants, thereby removing the restriction on the number of participants in federated learning modeling. More than two participants can also realize federated learning modeling through this solution, which improves the flexibility of federated learning modeling.

[0087] Figure 3 The flowchart of a model training method based on federated learning provided by an embodiment of the present application is schematically shown, and the method can be implemented by participants in the federated learning calculation. Figure 3 As shown, the model training method based on federated learning provided in this embodiment includes steps 310 to 380, which are as follows:

[0088] Step 310: Obtain a confusion factor data set and a confusion commitment of a preset data length corresponding to the current participant sent by the coordinator; wherein the confusion factor data set and the confusion commitment are generated by the coordinator based on sample data and the number of participants provided by multiple participants; the confusion factor data set includes multiple confusion factors of preset data lengths, the number of confusion factors is the same as the number of sample data provided by the multiple participants, and the confusion commitment corresponding to the current participant is generated based on the confusion factor corresponding to the sample data of the current participant.

[0089] Step 320: Perform data encryption processing according to the obfuscation factor data set and the obfuscation commitment to obtain the encrypted data of the current participant.

[0090] Step 310 - Step 320 may refer to the description of step 210 - Step 220 in the aforementioned embodiment, which will not be repeated here.

[0091] Step 330: Calculate based on the encrypted data and model parameters of the current participant to obtain a first secret parameter shard; wherein the model parameters have initial values; the first secret parameter shard represents the predicted value of the model for the sample data.

[0092] Specifically, the sample data provided by each participant includes training data and label data. The training data is equivalent to the input data of the model, and the label data is equivalent to the label corresponding to the output data of the model. The output data of the model is the prediction data of the model for the input data. Based on the type of sample data, the corresponding encrypted data is also divided into data ciphertext shards corresponding to the training data and data ciphertext shards corresponding to the label data. When performing calculations, the label data does not directly participate in the calculations. The training data in the encrypted data of the current participant is used as the model input data, and the model parameters are calculated. The model prediction value obtained is used as the first secret parameter shard. The current participant secretly saves the first secret parameter shard locally. For example, if the data y, x1, x2, and x3 provided by the aforementioned participants, data y belongs to the label data, and data x1, x2, and x3 all belong to the training data, then correspondingly, the data ciphertext shard share_y corresponding to data y belongs to the data ciphertext shard corresponding to the label data, and the data ciphertext shards share_x1, share_x2, and share_x3 corresponding to data x1, x2, and x3 belong to the data ciphertext shard corresponding to the training data.

[0093] In one embodiment of the present application, the model parameters include a first model parameter and a second model parameter, and the calculation process specifically includes: obtaining a data feature summation slice of the current participant based on the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant; multiplying the data feature summation slice of the current participant by the first model parameter, and adding the second model parameter to obtain a first secret parameter slice.

[0094] Specifically, the data ciphertext slices corresponding to all training data are summed to obtain the data feature sum slice. For example, in the above example, the data feature sum slice share_1_feature_sum of participant 1 is

[0095] share_x1+share_x2+share_x3. Then the data feature sum shard is calculated with the model parameters to obtain the first cryptographic parameter shard. The federated learning model in this embodiment adopts a logistic regression algorithm, and its expression is G=W*x+b, where x represents the input data, that is, the data feature sum shard, W and b are both model parameters, W represents the first model parameter, and b represents the second model parameter. Then the first secret parameter shard G_share_i=W*share_i_feature_sum+b of the current participant (the i-th participant), for example, the first secret parameter shard G_share_1=W*share_1_feature_sum+b of participant 1. In the first calculation, the first model parameter W and the second model parameter b are both initial values, and the two model parameters will be updated in subsequent calculations.

[0096] In some cases, after the logistic regression algorithm, normalization or nonlinear processing is required. Then G = W*x + b does not directly represent the model prediction value. The model prediction value is further calculated based on the value of the logistic regression algorithm. For example, the model prediction value ypred = 1 / (1+e -G ).

[0097] Step 340: Perform multiple ciphertext multiplication operations on the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter slice; wherein the second secret parameter slice represents the characteristic value of the model after processing the sample data.

[0098] Specifically, the ciphertext multiplication operation refers to the multiplication operation of two data in the form of encryption processing. Each ciphertext multiplication operation has two input data, which can be the same or different. When multiple ciphertext multiplication operations are performed based on the encrypted data of the current participant and the first secret parameter shard, the output value of the previous ciphertext multiplication operation is used as the input data of the next ciphertext multiplication operation, and finally the second secret parameter shard is obtained, which represents the feature value of the model after processing the sample data. The current participant secretly saves the second secret parameter shard locally.

[0099] The following uses two different input data as an example to illustrate the specific process of a ciphertext multiplication operation. Assuming that the ciphertext multiplication operation of the first input data T and the second input data P is to be performed, the output value of the corresponding ciphertext multiplication operation is Z, then it can be recorded as: Z = T x P. The specific process is as follows:

[0100] S1. First, the participant initiates a request to the coordinator based on the input data participating in the ciphertext multiplication operation. After receiving the request, the coordinator randomly generates a first multiplication confusion factor o_mul_first and a second multiplication confusion factor o_mul_second of a preset length (for example, 256 bits), and uses the product of the first multiplication confusion factor and the second multiplication confusion factor as the third multiplication confusion factor o_mul_third, that is, o_mul_third = o_mul_first*o_mul_second.

[0101] S2. The coordinator generates a corresponding multiplication confusion factor slice for each participant for each multiplication confusion factor according to the above three multiplication confusion factors and the number of participants. The sum of these multiplication confusion factor slices is the corresponding multiplication confusion factor. For example, for the first multiplication confusion factor o_mul_first, assuming that there are 4 participants, 4 first multiplication confusion factor slices are generated: o_mul_first_1, o_mul_first_2, o_mul_first_3, o_mul_first_4, o_mul_first=o_mul_first_1+o_mul_first_2+o_mul_first_3+o_mul_first_4. Similarly, the second multiplication confusion factor o_mul_second corresponds to 4 second multiplication confusion factor slices, and the third multiplication confusion factor o_mul_third corresponds to 4 third multiplication confusion factor slices.

[0102] S3. The coordinator sends the three multiplication confusion factor slices corresponding to each participant to each participant, so that each participant includes three multiplication confusion factor slices. For example, participant 1 includes the first multiplication confusion factor slice o_mul_first_1, the second multiplication confusion factor slice o_mul_second_1 and the third multiplication confusion factor slice o_mul_third_1.

[0103] S4. The participants calculate the intermediate shards based on the input data participating in the ciphertext multiplication operation and the above-mentioned multiplication confusion factor, where there are two input data and two corresponding intermediate shards. The first intermediate shard is the difference between the first input data and the first multiplication confusion factor shard, and the second intermediate shard is the difference between the second input data and the second multiplication confusion factor shard, that is, the first intermediate shard t_i = T-o_mul_first_i, the second intermediate shard p_i = P-o_mul_second_i, and i represents the number of the participant. For example, for participant 1, the first intermediate shard t_1 = T-o_mul_first_1, and the second intermediate shard p_1 = P-o_mul_second_1.

[0104] S5. The participants send the above two intermediate shards to the coordinator, who counts the two intermediate shards of all the participants respectively, obtains the corresponding summary shards, and sends the two summary shards to each participant. Specifically, the sum of the first intermediate shards of each participant is counted to obtain the first summary shard; the sum of the second intermediate shards of each participant is counted to obtain the second summary shard. Taking 4 participants as an example, the first summary shard t = t_1 + t_2 + t_3 + t_4, the second summary shard p = p_1 + p_2 + p_3 + p_4, and finally participants 1 to 4 all obtain the first summary shard t and the second summary shard p.

[0105] S6. The participants use the above three multiplication confusion factor slices and the above two summary slices to calculate the output value of the ciphertext multiplication operation. The calculation process specifically includes: multiplying the first multiplication confusion factor slice and the first summary slice to obtain a first product; multiplying the second multiplication confusion factor slice and the second summary slice to obtain a first product; multiplying the first summary slice and the second summary slice to obtain a third product; taking the sum of the first product, the second product, the third product and the third multiplication confusion factor slice as the output value of the ciphertext multiplication operation. That is, the output value of the ciphertext multiplication operation z_i=TxP=o_mul_third_i+o_mul_second_i*t+o_mul_first_i*p+t*p, where i represents the number of the participant. At this point, a ciphertext multiplication operation is completed.

[0106] In one embodiment of the present application, the process of performing multiple ciphertext multiplication operations based on the encrypted data of the current participant and the first secret parameter slice specifically includes: performing ciphertext multiplication operations on the data ciphertext slice corresponding to the label data in the encrypted data of the current participant and the first secret parameter slice to obtain the second secret parameter stage one slice; multiplying the second secret parameter stage one slice by the first set coefficient to obtain the second secret parameter stage two slice; performing ciphertext multiplication operations on the data ciphertext slice corresponding to the label data in the encrypted data of the current participant and the second secret parameter stage two slice to obtain the second secret parameter stage three slice; subtracting the product of the data ciphertext slice corresponding to the label data and the second set coefficient from the second secret parameter stage three slice to obtain the second secret parameter slice.

[0107] Specifically, based on the specific process of the above-mentioned ciphertext multiplication operation, the data ciphertext fragment share_y_i corresponding to the label data in the encrypted data of the current participant is used as the first input data, and the first secret parameter fragment G_share_i is used as the second input data, and the above-mentioned S1-S6 ciphertext multiplication operations are performed. The output value of the ciphertext multiplication operation obtained represents the second secret parameter stage one fragment yG_share_i, that is, yG_share_i=G_share_iⅹshare_y_i, where i represents the number of the participant.

[0108] Then, the second secret parameter phase one shard is multiplied by the first setting coefficient. In this embodiment, the first setting coefficient is 0.25, so the second secret parameter phase two shard yG_offset_share_i=0.25*yG_share_i, where i represents the number of the participant.

[0109] Next, the data ciphertext fragment share_y_i corresponding to the tag data in the encrypted data of the current participant is used as the first input data, and the second secret parameter stage two fragment yG_offset_share_i is used as the second input data, and the above-mentioned ciphertext multiplication operation S1-S6 is performed. The output value of the ciphertext multiplication operation obtained represents the second secret parameter stage three fragment d_left_share_i, that is, d_left_share_i=yG_offset_share_iⅹshare_y_i, where i represents the number of the participant.

[0110] Finally, the data ciphertext fragment share_y_i corresponding to the tag data in the encrypted data of the current participant is multiplied by the second set coefficient to obtain a value, and the second secret parameter stage three fragment d_left_share_i is subtracted from the value to obtain the second secret parameter fragment d_share_i. In this embodiment, the second set coefficient is 0.5, that is, d_share_i=d_left_share_i-0.5*share_y_i, where i represents the number of the participant.

[0111] Step 350: Perform a ciphertext multiplication operation on the encrypted data of the current participant and the second secret parameter shard to obtain a third secret parameter shard; wherein the third secret parameter shard represents the model gradient.

[0112] Specifically, the encrypted data of the current participant includes multiple data, and the ciphertext multiplication operation mainly uses training data. Therefore, the process of the ciphertext multiplication operation specifically includes: according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant, the data feature sum slice of the current participant is obtained; according to the data feature sum slice and the second secret parameter slice, the ciphertext multiplication operation is performed to obtain the third secret parameter slice. That is, first obtain the data feature sum slice share_i_feature_sum, and then use the data feature sum slice share_i_feature_sum as the first input data and the second secret parameter slice d_share_i as the second input data to perform S1-S6 of the aforementioned ciphertext multiplication operation, and the output value of the ciphertext multiplication operation obtained represents the third secret parameter slice grad_share_i, that is, grad_share_i=d_share_iⅹshare_i_feature_sum, where i represents the number of the participant. The third secret parameter slice represents the model gradient grad.

[0113] Step 360: Calculate a fourth secret parameter slice based on the first secret parameter slice; wherein the second secret parameter slice represents the model loss.

[0114] Specifically, the fourth secret parameter shard is used to reflect the model loss, which also includes the ciphertext multiplication operation.

[0115] In one embodiment of the present application, the calculation process of the fourth secret parameter shard includes: performing a ciphertext multiplication operation according to the first secret parameter shard to obtain a fourth secret parameter stage one shard; multiplying the fourth secret parameter stage one shard by a third set coefficient to obtain a fourth secret parameter stage two shard; subtracting the product of the second secret parameter stage one shard and the fourth set coefficient from the fourth secret parameter stage two shard to obtain the fourth secret parameter shard.

[0116] First, the first secret parameter shards are respectively the first input data and the second input data, that is, the first input data and the second input data are the same, both are the first secret parameter shard G_share_i, and the above-mentioned ciphertext multiplication operations S1-S6 are performed. The output value of the ciphertext multiplication operation obtained represents the fourth secret parameter stage one shard loss_g_squard_share_i, that is, loss_g_squard_share_i=G_share_iⅹG_share_i, wherein i represents the number of the participant.

[0117] Then, the fourth secret parameter stage one shard is multiplied by the third setting coefficient to obtain the fourth secret parameter stage two shard loss_g_squard_offset_share_i. In this embodiment, the third setting coefficient is 0.125, that is, loss_g_squard_offset_share_i=0.125*loss_g_squard_share_i, where i represents the number of the participant.

[0118] Finally, the second secret parameter stage one shard yG_share_i is multiplied by the fourth setting coefficient to obtain a value, and the value is subtracted from the fourth secret parameter stage two shard loss_g_squard_offset_share_i to obtain the fourth secret parameter shard loss_share_i. In this embodiment, the fourth setting coefficient is 0.5, that is, loss_share_i=loss_g_squard_offset_share_i-0.5*yG_share_i, where i represents the number of the participant.

[0119] Step 370: Use the first secret parameter slice, the second secret parameter slice, the third secret parameter slice and the fourth secret parameter slice as the calculation results of the current participant.

[0120] Specifically, the first secret parameter slice, the second secret parameter slice, the third secret parameter slice and the fourth secret parameter slice calculated by the above process are all calculation results of the participants. Each participant can summarize the above calculation results and send them to the coordinator, or can send the result to the coordinator every time one of the results is calculated.

[0121] Step 380: Send the calculation result of the current participant to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and adjust the model parameters according to the summary calculation result.

[0122] For step 380, reference may be made to the relevant description in the aforementioned step 240, which will not be repeated here.

[0123] In the technical solution provided in the embodiments of the present application, data ciphertext shards are obtained by encrypting the data according to the confusion factor and confusion commitment generated by the coordinating party, and ciphertext multiplication operations are performed based on the data ciphertext shards to obtain the final calculation results. Finally, the calculation results are summarized by the coordinating party to adjust the model parameters, thereby realizing multi-party logistic regression calculations. The participating parties are allowed to rely on the coordinating party to achieve efficient multi-party logistic regression calculations without leaking data privacy, so that federated learning modeling can more flexibly support the participation of any number of parties; and as the number of participating parties increases, the network complexity does not change significantly, which is more friendly to federated learning modeling in multi-party scenarios.

[0124] Figure 4 A schematic diagram of a model training method based on federated learning provided by an embodiment of the present application is schematically shown. Figure 4 The specific implementation process of the model training method based on federated learning provided in this application is described by taking two participants as an example. Figure 4 As shown, the process of the model training method based on federated learning provided by the embodiment of the present application includes: participant i makes a ciphertext data shard list for data x. Participant i sends the ciphertext data shard list to the coordinator. Participant j requests the ciphertext data shard xj from the coordinator. Participants i and j perform ciphertext calculations respectively. Participant i exchanges ciphertext calculation shards with the coordinator. Participant j requests the ciphertext calculation shards from the coordinator. Participants i and j respectively summarize the ciphertext calculation result shards to the coordinator. The coordinator publishes the summary results, that is, sends the summary results to participants i and j respectively. The specific implementation process of the above content has been described in the aforementioned embodiments and will not be repeated here.

[0125] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0126] The following introduces an apparatus embodiment of the present application, which can be used to execute the model training method based on federated learning in the above-mentioned embodiment of the present application. Figure 5 The structural block diagram of the model training device based on federated learning provided in the embodiment of the present application is schematically shown. Figure 5 As shown, the model training device based on federated learning provided in the embodiment of the present application includes:

[0127] The obfuscation data acquisition module 510 is used to acquire the obfuscation factor data group and obfuscation commitment of the preset data length corresponding to the current participant sent by the coordinator; wherein the obfuscation factor data group and obfuscation commitment are generated by the coordinator based on the sample data and the number of participants provided by multiple participants; the obfuscation factor data group includes multiple obfuscation factors of the preset data length, the number of the obfuscation factors is the same as the number of sample data provided by multiple participants, and the obfuscation commitment corresponding to the current participant is generated according to the obfuscation factor corresponding to the sample data of the current participant;

[0128] An encryption processing module 520 is used to perform data encryption processing according to the obfuscation factor data group and the obfuscation commitment to obtain the encrypted data of the current participant;

[0129] A model calculation module 530, configured to perform model calculation based on the encrypted data of the current participant to obtain a calculation result of the current participant;

[0130] The parameter adjustment module 540 is used to send the calculation result of the current participant to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and adjust the model parameters according to the summary calculation result.

[0131] In one embodiment of the present application, the number of the confusion factor data groups is the same as the number of the participants; the confusion factor data group includes multiple randomly generated confusion factors of preset data lengths, and one confusion factor corresponds to one sample data; the confusion commitment corresponding to the current participant is the sum of the multiple confusion factors corresponding to the sample data of the current participant.

[0132] In one embodiment of the present application, the encrypted data of the current participant includes multiple ciphertext data fragments; the encryption processing module 520 is specifically used to:

[0133] Traversing the confusion factors in the confusion factor data group, if the sample data corresponding to the current confusion factor does not belong to the current participant, using the current confusion factor as the data ciphertext fragment of the sample data at the current participant;

[0134] If the sample data corresponding to the current confusion factor belongs to the current participant, the sample data is subtracted from the confusion commitment and then added with the current confusion factor as the ciphertext data fragment of the sample data at the current participant.

[0135] In one embodiment of the present application, the model calculation module 530 includes:

[0136] A first calculation unit is used to calculate according to the encrypted data of the current participant and the model parameters to obtain a first secret parameter shard; wherein the model parameters have initial values; and the first secret parameter shard represents the predicted value of the model for the sample data;

[0137] A second calculation unit is used to perform multiple ciphertext multiplication operations on the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter slice; wherein the second secret parameter slice represents a characteristic value after the model processes the sample data;

[0138] A third calculation unit is used to perform a ciphertext multiplication operation on the encrypted data of the current participant and the second secret parameter slice to obtain a third secret parameter slice; wherein the third secret parameter slice represents a model gradient;

[0139] a fourth calculation unit, configured to calculate a fourth secret parameter slice according to the first secret parameter slice; wherein the second secret parameter slice represents a model loss;

[0140] A result generating unit is used to use the first secret parameter slice, the second secret parameter slice, the third secret parameter slice and the fourth secret parameter slice as the calculation result of the current participant.

[0141] In one embodiment of the present application, the sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the training data; the model parameters include the first model parameters and the second model parameters; the first calculation unit is specifically used to:

[0142] Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant;

[0143] The product of the data feature sum slice of the current participant and the first model parameter is added to the second model parameter to obtain the first secret parameter slice.

[0144] In one embodiment of the present application, the sample data provided by the multiple participants include label data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the label data; and the second computing unit is specifically used to:

[0145] Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter stage one slice;

[0146] Multiplying the second secret parameter stage one slice by the first set coefficient to obtain the second secret parameter stage two slice;

[0147] Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the second secret parameter stage two slice to obtain the second secret parameter stage three slice;

[0148] The second secret parameter stage three slice is subtracted from the product of the data ciphertext slice corresponding to the tag data and the second set coefficient to obtain the second secret parameter slice.

[0149] In one embodiment of the present application, the second computing unit is specifically configured to:

[0150] Obtaining a first multiplication confusion factor slice, a second multiplication confusion factor slice, and a third multiplication confusion factor slice corresponding to the current participant; wherein each multiplication confusion factor slice is generated by slicing the corresponding multiplication confusion factor according to the number of participants, the first multiplication confusion factor and the second multiplication confusion factor are data of the preset data length randomly generated by the coordinator, and the third multiplication confusion factor is the product of the first multiplication confusion factor and the second multiplication confusion factor;

[0151] Obtaining a first intermediate shard of the current participant according to a difference between the first secret parameter shard and the first multiplication confusion factor shard, and obtaining a second intermediate shard of the current participant according to a difference between a data ciphertext shard corresponding to the tag data and the second multiplication confusion factor shard;

[0152] Sending the first intermediate shard and the second intermediate shard to the coordinator to obtain a first summary shard and a second summary shard fed back by the coordinator, wherein the first summary shard is obtained by the coordinator summarizing the first intermediate shards of multiple participants, and the second summary shard is obtained by the coordinator summarizing the second intermediate shards of multiple participants;

[0153] The second secret parameter phase one slice is generated according to the first multiplication confusion factor slice, the second multiplication confusion factor slice, the third multiplication confusion factor slice, the first summary slice, and the second summary slice.

[0154] In one embodiment of the present application, the second computing unit is specifically configured to:

[0155] Multiplying the first multiplication confusion factor slice and the first summary slice to obtain a first product;

[0156] Multiplying the second multiplication confusion factor slice and the second summary slice to obtain a first product;

[0157] multiplying the first summary slice and the second summary slice to obtain a third product;

[0158] The sum of the first product, the second product, the third product and the third multiplication confusion factor shard is used as the second secret parameter stage one shard.

[0159] In one embodiment of the present application, the sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the training data; and the third computing unit is specifically used to:

[0160] Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant;

[0161] A ciphertext multiplication operation is performed on the data feature sum slice and the second secret parameter slice to obtain the third secret parameter slice.

[0162] In one embodiment of the present application, the fourth computing unit is specifically configured to:

[0163] Performing a ciphertext multiplication operation on the first secret parameter slice to obtain a fourth secret parameter stage one slice;

[0164] Multiplying the fourth secret parameter stage one slice by a third set coefficient to obtain a fourth secret parameter stage two slice;

[0165] The fourth secret parameter stage two slice is subtracted from the product of the second secret parameter stage one slice and the fourth setting coefficient to obtain the fourth secret parameter slice.

[0166] The specific details of the federated learning-based model training device provided in each embodiment of the present application have been described in detail in the corresponding method embodiments and will not be repeated here.

[0167] Figure 6 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0168] It should be noted that Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0169] like Figure 6 As shown, the computer system 600 includes a central processing unit 601 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 602 (ROM) or the program loaded from the storage part 608 to the random access memory 603 (RAM). Various programs and data required for system operation are also stored in the random access memory 603. The central processing unit 601, the read-only memory 602 and the random access memory 603 are connected to each other through a bus 604. An input / output interface 605 (Input / Output interface, i.e., I / O interface) is also connected to the bus 604.

[0170] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a local area network card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0171] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processor 601, various functions defined in the system of the present application are executed.

[0172] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0173] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0175] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0176] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0177] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A model training method based on federated learning, characterized in that: include: Obtaining a confusion factor data set and a confusion commitment of a preset data length corresponding to the current participant sent by the coordinator; wherein the confusion factor data set and the confusion commitment are generated by the coordinator based on sample data and the number of participants provided by multiple participants; the confusion factor data set includes multiple confusion factors of preset data lengths, the number of the confusion factors is the same as the number of sample data provided by multiple participants, and the confusion commitment corresponding to the current participant is generated according to the confusion factor corresponding to the sample data of the current participant; Performing data encryption processing according to the obfuscation factor data group and the obfuscation commitment to obtain encrypted data of the current participant; Performing model calculation based on the encrypted data of the current participant to obtain a calculation result of the current participant; The calculation result of the current participant is sent to the coordinator to obtain the summary calculation result fed back by the coordinator based on the calculation results of multiple participants, and the model parameters are adjusted according to the summary calculation result.

2. The model training method based on federated learning according to claim 1, characterized in that: The number of the confusion factor data groups is the same as the number of the participants; the confusion factor data group includes multiple randomly generated confusion factors of preset data lengths, and one confusion factor corresponds to one sample data; the confusion commitment corresponding to the current participant is the sum of the multiple confusion factors corresponding to the sample data of the current participant.

3. The model training method based on federated learning according to claim 1, characterized in that: The encrypted data of the current participant includes multiple ciphertext data fragments; Performing data encryption processing according to the obfuscation factor data group and the obfuscation commitment to obtain the encrypted data of the current participant, including: Traversing the confusion factors in the confusion factor data group, if the sample data corresponding to the current confusion factor does not belong to the current participant, using the current confusion factor as the data ciphertext fragment of the sample data at the current participant; If the sample data corresponding to the current confusion factor belongs to the current participant, the sample data is subtracted from the confusion commitment and then added with the current confusion factor as the ciphertext data fragment of the sample data at the current participant.

4. The model training method based on federated learning according to claim 1, characterized in that: Performing model calculation based on the encrypted data of the current participant to obtain the calculation result of the current participant includes: Calculating according to the encrypted data of the current participant and the model parameters to obtain a first secret parameter shard; wherein the model parameters have initial values; the first secret parameter shard represents the predicted value of the model for the sample data; Perform multiple ciphertext multiplication operations on the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter slice; wherein the second secret parameter slice represents the characteristic value of the model after processing the sample data; Performing a ciphertext multiplication operation on the encrypted data of the current participant and the second secret parameter slice to obtain a third secret parameter slice; wherein the third secret parameter slice represents a model gradient; A fourth secret parameter slice is calculated based on the first secret parameter slice; wherein the second secret parameter slice represents the model loss; The first secret parameter slice, the second secret parameter slice, the third secret parameter slice and the fourth secret parameter slice are used as calculation results of the current participant.

5. The model training method based on federated learning according to claim 4, characterized in that: The sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the training data; the model parameters include the first model parameters and the second model parameters; Calculating according to the encrypted data of the current participant and the model parameters to obtain a first secret parameter fragment, including: Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant; The product of the data feature sum slice of the current participant and the first model parameter is added to the second model parameter to obtain the first secret parameter slice.

6. The model training method based on federated learning according to claim 4, characterized in that: The sample data provided by the multiple participants include label data; the encrypted data of the current participant includes the data ciphertext fragment corresponding to the label data; Performing multiple ciphertext multiplication operations on the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter slice includes: Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter stage one slice; Multiplying the second secret parameter stage one slice by the first set coefficient to obtain the second secret parameter stage two slice; Perform a ciphertext multiplication operation on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the second secret parameter stage two slice to obtain the second secret parameter stage three slice; The second secret parameter stage three slice is subtracted from the product of the data ciphertext slice corresponding to the tag data and the second set coefficient to obtain the second secret parameter slice.

7. The model training method based on federated learning according to claim 6, characterized in that: A ciphertext multiplication operation is performed on the data ciphertext slice corresponding to the tag data in the encrypted data of the current participant and the first secret parameter slice to obtain a second secret parameter stage one slice, including: Obtaining a first multiplication confusion factor slice, a second multiplication confusion factor slice, and a third multiplication confusion factor slice corresponding to the current participant; wherein each multiplication confusion factor slice is generated by slicing the corresponding multiplication confusion factor according to the number of participants, the first multiplication confusion factor and the second multiplication confusion factor are data of the preset data length randomly generated by the coordinator, and the third multiplication confusion factor is the product of the first multiplication confusion factor and the second multiplication confusion factor; Obtaining a first intermediate shard of the current participant according to a difference between the first secret parameter shard and the first multiplication confusion factor shard, and obtaining a second intermediate shard of the current participant according to a difference between a data ciphertext shard corresponding to the tag data and the second multiplication confusion factor shard; Sending the first intermediate shard and the second intermediate shard to the coordinator to obtain a first summary shard and a second summary shard fed back by the coordinator, wherein the first summary shard is obtained by the coordinator summarizing the first intermediate shards of multiple participants, and the second summary shard is obtained by the coordinator summarizing the second intermediate shards of multiple participants; The second secret parameter phase one slice is generated according to the first multiplication confusion factor slice, the second multiplication confusion factor slice, the third multiplication confusion factor slice, the first summary slice, and the second summary slice.

8. The model training method based on federated learning according to claim 6, characterized in that: Generating the second secret parameter phase one slice according to the first multiplication confusion factor slice, the second multiplication confusion factor slice, the third multiplication confusion factor slice, the first summary slice, and the second summary slice, comprising: Multiplying the first multiplication confusion factor slice and the first summary slice to obtain a first product; Multiplying the second multiplication confusion factor slice and the second summary slice to obtain a first product; multiplying the first summary slice and the second summary slice to obtain a third product; The sum of the first product, the second product, the third product and the third multiplication confusion factor shard is used as the second secret parameter stage one shard.

9. The model training method based on federated learning according to claim 6, characterized in that: The sample data provided by the multiple participants include training data; the encrypted data of the current participant includes the data ciphertext fragments corresponding to the training data; Performing a ciphertext multiplication operation on the encrypted data of the current participant and the second secret parameter slice to obtain a third secret parameter slice includes: Obtaining a data feature summed slice of the current participant according to the sum of the data ciphertext slices corresponding to the training data in the encrypted data of the current participant; A ciphertext multiplication operation is performed on the data feature sum slice and the second secret parameter slice to obtain the third secret parameter slice.

10. The model training method based on federated learning according to claim 6, characterized in that: Calculating a fourth secret parameter slice according to the first secret parameter slice includes: Performing a ciphertext multiplication operation on the first secret parameter slice to obtain a fourth secret parameter stage one slice; Multiplying the fourth secret parameter stage one slice by a third set coefficient to obtain a fourth secret parameter stage two slice; The fourth secret parameter stage two slice is subtracted from the product of the second secret parameter stage one slice and the fourth setting coefficient to obtain the fourth secret parameter slice.