Model training methods for vertical federated learning and horizontal federated learning

Abstract

The present invention relates to the field of financial technology, and discloses a sample prediction method, a model training method for vertical federated learning and a horizontal federated learning method. The prediction method comprises: obtaining a target feature vector corresponding to a user's sample, and inputting the target feature vector into a prediction model, wherein the prediction model is trained by a vertical federated learning method and a horizontal federated learning method using a first sample of a plurality of guest terminals and a second sample of a plurality of host terminals; obtaining a value output by the prediction model, and determining a prediction result of the user according to the value. The user risk prediction accuracy of the present invention is high.

Description

Technical Field

[0001] The present invention relates to the technical field of financial technology (Fintech), and in particular to a model training method for vertical federated learning and horizontal federated learning. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] User data is distributed in different institutions, forming data islands. Federated learning is a good way to solve data islands. At present, there are two common federated modeling methods according to different data distribution situations. One is that the two parties have a large number of overlapping users, but the features are different. In this case, vertical federated modeling can be used to increase the available features. Another scenario is that the two parties have a large number of overlapping features, but the user IDs are different. Horizontal federated modeling can be used to increase the number of available samples. In the field of financial risk control, user risks can be predicted through the model established by federated learning.

[0004] In reality, there are often multiple institutions that want to unite with each other, some of which are suitable for vertical federation and some for horizontal federation.

[0005] Taking the federated modeling of banks and Internet companies as an example, there may be a situation where multiple banks jointly model with an Internet company. The features and label information between the Guest terminals of multiple banks (the Guest terminal is a terminal with sample labels) are highly consistent, but the number of features of the samples in the Guest terminal of the bank is too small. The effect of directly using the data of the Guest terminal of the bank and the data of the Host terminal of the Internet company (the Host terminal is a terminal without sample labels) for horizontal joint modeling is not good, resulting in low accuracy in predicting user risks by the established model. The labels of the data in the bank are all classified, such as good users / bad users. The features available to the bank itself include credit data, deposits and transaction data, etc. If the data of different banks are used for vertical federated modeling, the established model will lack the support of Internet data that represents user behavior habits, which will make the model predict user risks with low accuracy. It can be seen from this that in the prior art, there is a problem of low accuracy in predicting user risks. Summary of the invention

[0006] The main purpose of the present invention is to provide a model training method for vertical federated learning and horizontal federated learning, aiming to solve the problem of low accuracy in user risk prediction.

[0007] In a first aspect, the present invention provides a prediction method, the prediction method comprising:

[0008] Obtain a target feature vector corresponding to the user's sample, and input the target feature vector into a prediction model, wherein the prediction model is trained by using a plurality of first samples of the guest end and a plurality of second samples of the host end through a vertical federated learning method and a horizontal federated learning method;

[0009] The numerical value output by the prediction model is obtained, and the prediction result of the user is determined according to the numerical value.

[0010] In a second aspect, the present invention further provides a model training method for vertical federated learning and horizontal federated learning, which is applied to a guest end, and the model training method for vertical federated learning and horizontal federated learning includes:

[0011] The Guest end receives each first parameter sent by the Host end and an identifier corresponding to the first parameter, and determines the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the Host end and the corresponding weight of the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter;

[0012] Determine a third parameter based on the first parameter and the second parameter of the same identifier, and determine a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter, so as to complete vertical federated learning;

[0013] determining an intermediate parameter according to the first gradient, and sending the intermediate parameter to the host end, so that the host end determines a second gradient of the host end according to the intermediate parameter and each of the first eigenvectors;

[0014] Determine first sending information according to the first gradient and the first number of the second samples, and send the first sending information to a third-party terminal, wherein the third-party terminal determines the first model parameter corresponding to the guest according to the first gradient and the first number sent by each guest, the second gradient sent by each host, and the second number of the first samples, so as to complete horizontal federated learning;

[0015] The first model parameters fed back by the third-party terminal are received, and the parameters of the first model are updated according to the first model parameters, and the first model with updated parameters is trained according to each of the second samples to obtain a first prediction model.

[0016] In a third aspect, the present invention provides a model training method for vertical federated learning and horizontal federated learning, which is applied to a host end. The model training method for vertical federated learning and horizontal federated learning includes:

[0017] Determine each first sample from each sample in the Host end, and determine a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector;

[0018] Determine a first parameter according to the first feature vector and a weight corresponding to the first feature vector, and send each of the first parameters and an identifier of a first sample corresponding to the first parameter to the guest end;

[0019] receiving the intermediate parameters fed back by the guest, and determining a second gradient according to the intermediate parameters and each of the first feature vectors to complete the vertical federated learning;

[0020] Generate second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals, so as to complete horizontal federated learning;

[0021] The second model parameters fed back by the third terminal are received, and the parameters of the second model are updated according to the second model parameters, and the second model with updated parameters is trained according to each of the first samples to obtain a second prediction model.

[0022] In a third aspect, the present invention provides a model training method for vertical federated learning and horizontal federated learning, which is applied to a third-party terminal. The model training method for vertical federated learning and horizontal federated learning includes:

[0023] Obtaining first transmission information sent by each guest end and second transmission information sent by each host end, and obtaining a first gradient and a first number of second samples according to the first transmission information, and obtaining a second gradient and a second number of first samples according to the second transmission information;

[0024] Determine a model parameter to be determined according to the first gradient, the first quantity, the second gradient, and the second quantity;

[0025] Generate a first model parameter corresponding to the Guest end according to the model parameter to be processed and the first number corresponding to the Guest end, and generate a second model parameter corresponding to the Host end according to the model parameter to be processed and the second number corresponding to the Host end;

[0026] The first model parameter is sent to a Guest end corresponding to the first model parameter, and the second model parameter is sent to a Host end corresponding to the second model parameter.

[0027] In a fifth aspect, the present invention provides a prediction terminal, the prediction terminal comprising:

[0028] A first acquisition module is used to acquire a target feature vector corresponding to a user's sample and input the target feature vector into a prediction model, wherein the prediction model is trained by using a plurality of first samples of guest terminals and a plurality of second samples of host terminals through a vertical federated learning method and a horizontal federated learning method;

[0029] The first acquisition module is further used to obtain the numerical value output by the prediction model and determine the prediction result of the user according to the numerical value.

[0030] In a sixth aspect, the present invention provides a Guest terminal, wherein the Guest terminal comprises:

[0031] A first receiving module is used to receive each first parameter sent by the Host end and an identifier corresponding to the first parameter, and determine the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the Host end and the weight corresponding to the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter;

[0032] A first determination module, configured to determine a third parameter based on the first parameter and the second parameter of the same identifier, and determine a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and a label value corresponding to the third parameter, so as to complete vertical federated learning;

[0033] The first determination module is further configured to determine an intermediate parameter according to the first gradient, and send the intermediate parameter to the Host end, so that the Host end determines a second gradient of the Host end according to the intermediate parameter and each of the first eigenvectors;

[0034] A first sending module, configured to determine first sending information according to the first gradient and the first number of the second samples, and send the first sending information to a third-party terminal, wherein the third-party terminal determines a first model parameter corresponding to the guest according to the first gradient and the first number sent by each guest, the second gradient sent by each host, and the second number of the first samples, so as to complete horizontal federated learning;

[0035] The first receiving module is further used to receive the first model parameters fed back by the third-party terminal, update the parameters of the first model according to the first model parameters, and train the first model with updated parameters according to each of the second samples to obtain a first prediction model.

[0036] In a seventh aspect, the present invention provides a host end, the host end comprising:

[0037] A second determination module, configured to determine each first sample from each sample in the Host end, and determine a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector;

[0038] The second determination module is used to determine the first parameter according to the first feature vector and the weight corresponding to the first feature vector, and send the first parameter and the identifier of the first sample corresponding to the first parameter to the guest end;

[0039] A second receiving module, configured to receive the intermediate parameters fed back by the guest, and determine a second gradient according to the intermediate parameters and each of the first feature vectors, so as to complete the vertical federated learning;

[0040] A second sending module, configured to generate second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals, so as to complete horizontal federated learning;

[0041] The second receiving module is further used to receive the second model parameters fed back by the third terminal, update the parameters of the second model according to the second model parameters, and train the second model with updated parameters according to each of the first samples to obtain a second prediction model.

[0042] In an eighth aspect, the present invention provides a third-party terminal, the third-party terminal comprising:

[0043] A second acquisition module, configured to acquire first transmission information sent by each guest end and second transmission information sent by each host end, and acquire a first gradient and a first number of second samples according to the first transmission information, and acquire a second gradient and a second number of first samples according to the second transmission information;

[0044] a third determining module, configured to determine a model parameter to be determined according to the first gradient, the first quantity, the second gradient, and the second quantity;

[0045] A generating module, configured to generate first model parameters corresponding to the Guest end according to the model parameters to be processed and the first quantity corresponding to the Guest end, and to generate second model parameters corresponding to the Host end according to the model parameters to be processed and the second quantity corresponding to the Host end;

[0046] The third sending module is used to send the first model parameter to the Guest end corresponding to the first model parameter, and send the second model parameter to the Host end corresponding to the second model parameter.

[0047] In a ninth aspect, the present invention provides an electronic device comprising a memory, a processor, and instructions or programs stored in the memory and executable on the processor, wherein the instructions or programs, when executed by the processor, implement the steps of the prediction method as described above or the steps of the model training method of vertical federated learning and horizontal federated learning as described above.

[0048] In a tenth aspect, the present invention provides a computer-readable storage medium, which includes a program or instructions, and when the instructions or program are executed by the processor, the steps of the prediction method as described above or the steps of the model training method of vertical federated learning and horizontal federated learning as described above are implemented.

[0049] In an eleventh aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the prediction method as described above or the steps of the model training method of vertical federated learning and horizontal federated learning as described above.

[0050] The present invention provides a prediction method, a model training method for vertical federated learning and horizontal federated learning. The prediction terminal obtains a prediction model by training a first sample on the guest side and a second sample on the host side in the scenarios of vertical federated learning and horizontal federated learning. The prediction terminal obtains a feature vector corresponding to the sample, and inputs the feature vector into the prediction model to obtain a numerical value output by the model, thereby determining the prediction result of the user according to the numerical value. In the present invention, the prediction model is trained by using multiple samples from the guest side and multiple samples from the host side under vertical federated learning and horizontal learning, and the use of multiple samples from the guest side ensures that the number of features of the samples of the training model is not too small, and the model is trained in combination with the samples from the guest side and the samples from multiple hosts so that the prediction result output by the prediction model is supported by the user behavior habit data, thereby improving the accuracy of the prediction result output by the prediction model, and the accuracy of user risk prediction is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1A flowchart of a prediction method provided by an embodiment of the present invention;

[0052] Figure 2 A flow chart of a model training method for vertical federated learning and horizontal federated learning provided for the implementation of the present invention;

[0053] Figure 3 Another flow chart of the model training method for vertical federated learning and horizontal federated learning provided for the implementation of the present invention;

[0054] Figure 4 Another flow chart of the model training method of vertical federated learning and horizontal federated learning provided for the implementation of the present invention;

[0055] Figure 5 A schematic diagram of terminal interactions involved in the model training method for vertical federated learning and horizontal federated learning provided in an embodiment of the present invention;

[0056] Figure 6 A structural diagram of a prediction terminal provided by an embodiment of the present invention;

[0057] Figure 7 A structural diagram of the Guest end provided by an embodiment of the present invention;

[0058] Figure 8 A structural diagram of the Host end provided in an embodiment of the present invention;

[0059] Fig. 9 A structural diagram of a third-party terminal provided in an embodiment of the present invention;

[0060] Fig.10 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 , Figure 1 The present invention provides a flow chart of a prediction method, wherein the prediction method comprises:

[0064] Step 101, obtaining a target feature vector corresponding to a user's sample, and inputting the target feature vector into a prediction model, wherein the prediction model is trained by using a plurality of first samples of guest terminals and a plurality of second samples of host terminals through a vertical federated learning method and a horizontal federated learning method;

[0065] In this embodiment, the execution subject is the prediction terminal. The prediction terminal can be a guest terminal or a host terminal. The guest terminal refers to a party having a sample and a label corresponding to the sample, while the host terminal refers to a party having only a sample but not a label of the sample. The guest terminal can be a terminal of a financial institution, for example, the guest terminal is a terminal of a bank. The host terminal can be a terminal of other types of companies other than a financial institution, for example, the host terminal is a terminal of an Internet company.

[0066] A prediction model is provided in the prediction terminal. If the prediction terminal is a guest terminal, the prediction model is a first prediction model; if the prediction terminal is a host terminal, the prediction model is a second prediction model. The prediction terminal obtains the prediction model by training multiple first samples of guest terminals and multiple second samples of host terminals in the scenarios of vertical federated learning and horizontal federated learning. There are multiple guest terminals used for training, so the number of features of the samples used for training the model is not too small. The number of features refers to the number of characteristic attributes of the sample. For example, the characteristic attributes are height, age, gender, and income, and the number of characteristic attributes is four, then the number of features of the sample is four. The second sample of the host terminal can be used to characterize the user's behavioral habits. Therefore, the prediction model predicts the user to obtain the prediction result, which is supported by the user's behavioral habit data, making the prediction result more accurate. The prediction type of the prediction model can be trained according to demand. For example, if a bank wants to predict whether a user will default on the loan, a prediction model that can be used to predict whether the user will default on the loan is trained.

[0067] When prediction is required, the user's sample is obtained, and the target feature vector corresponding to the sample is obtained. A feature vector refers to a feature of a sample, and the numerical value of the feature is represented by a feature vector. For example, if the user's income is a feature, the income is represented by a target feature vector. The user's sample corresponds to multiple target feature vectors. The prediction terminal inputs the target feature vector into the prediction model.

[0068] Step 102, obtaining the numerical value output by the prediction model, and determining the prediction result of the user according to the numerical value.

[0069] The prediction model is provided with weights w corresponding to different feature vectors, and the prediction model is provided with a sigmoid function. After the prediction model obtains the target feature vector x, sigmoid(w*x) is obtained based on the sigmoid function, w, and x. Sigmoid(w*x) is mapped to the interval (0,1), that is, the value corresponding to sigmoid(w*x) is a value within (0,1). The prediction terminal obtains the value output by the prediction model, and the prediction terminal is provided with a preset threshold value, which is greater than 0 and less than 1. The prediction terminal determines whether the value is greater than the preset threshold value. If the value is greater than the preset threshold value, the value is set to 1, indicating that the prediction result is a bad result. For example, if the prediction is whether the user's loan repayment is overdue, the value is 1, and the prediction result is that the user's loan repayment is overdue. If the value is less than the preset threshold value, the value is 0, indicating that the prediction result is a good result. For example, if the prediction is whether the user's loan repayment is overdue, the value is 0, and the prediction result is that the user's loan repayment is not overdue.

[0070] In the technical solution provided in this embodiment, the prediction terminal obtains a prediction model by training with the first sample of the guest end and the second sample of the host end in the scenarios of vertical federated learning and horizontal federated learning. The prediction terminal obtains the feature vector corresponding to the sample, and inputs the feature vector into the prediction model to obtain the numerical value output by the model, thereby determining the prediction result of the user according to the numerical value. The prediction model in the present invention is obtained by training with multiple samples of the guest end and multiple samples of the host end under vertical federated learning and horizontal learning, and the use of multiple samples of the guest end ensures that the number of features of the samples of the training model is not too small, and the model is trained in combination with the samples of the guest end and the samples of multiple hosts so that the prediction results output by the prediction model are supported by the user behavior habit data, thereby improving the accuracy of the prediction results output by the prediction model, and the accuracy of user risk prediction is high.

[0071] Reference Figure 2 , Figure 2 This is a flow chart of the model training method of vertical federated learning and horizontal federated learning of the present invention, which is applied to the Guest end. The method includes:

[0072] Step 201, the guest end receives each first parameter sent by the host end and an identifier corresponding to the first parameter, and determines a second parameter according to a second feature vector of a second sample corresponding to the identifier and a weight corresponding to the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the host end and the weight corresponding to the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter;

[0073] In this embodiment, several guests and several hosts participate in the training of the prediction model. The number of guests is the same as the number of hosts, and a guest is combined with a corresponding host to perform vertical federated learning first. For example, if there are three guests and three hosts, there are three combinations, one combination consists of one guest and one host, and the three combinations perform federated vertical learning at the same time.

[0074] Each guest terminal holds data samples with the same characteristics and labels with the same meaning; each host terminal holds data samples with the same characteristics.

[0075] When performing vertical federated learning, the Host end in the combination selects multiple first samples from the data samples, and extracts the first feature vector Xh corresponding to the first samples, and each first sample has a corresponding identifier, which can be the id; the Guest end in the combination selects multiple second samples from the data samples, and extracts the second feature vector Xg corresponding to the second samples, and each second sample has a corresponding identifier, which can be the id, and at the same time, each second sample corresponds to a label value y, which can be -1 or 1.

[0076] The sample corresponds to multiple features. For example, the features can be age, gender, and height, that is, a sample can extract multiple features, and each feature is represented by a feature vector. The features are set with corresponding weights, that is, the feature vectors are set with corresponding weights. After extracting the first feature vector corresponding to the first sample, the host side obtains the first parameter according to the first feature vector Xh and the weight Wh corresponding to the first feature vector, that is, Xh and Wh are combined to obtain the first parameter WXh. The host determines the identifier of the first sample corresponding to the first feature vector, and associates the identifier with the first parameter. For example, the serial number of the first sample is 1, and the identifier is defined as N1, then the first parameter WXh is associated with N1. The host side can calculate multiple first parameters, and then send each first parameter and the identifier associated with the first parameter to the guest side.

[0077] The Guest end is also provided with weights corresponding to the feature vectors. The Guest end selects multiple second samples from its own data samples, and extracts the second feature vector Wg corresponding to each second sample, and the second sample corresponds to multiple second feature vectors Wg. The Guest end obtains the second parameter based on the second feature vector Xg and the weight Wg corresponding to the second feature vector, that is, Xg and Wg are combined to obtain the second parameter WXg. In the process of calculating the second parameter, the Guest can receive the first parameters sent by the Host end and the identifiers associated with the first parameters.

[0078] Step 202, determining a third parameter based on the first parameter and the second parameter of the same identifier, and determining a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter, so as to complete vertical federated learning;

[0079] The second parameter is determined by the second feature vector corresponding to the second sample, and the identifier corresponding to the second sample can be regarded as the identifier corresponding to the second parameter. The Guest end determines the first parameter and the second parameter with the same identifier from the received identifier, and thus obtains the third parameter based on the first parameter and the second parameter. For example, the first parameter is WXh1, and the second parameter with the same identifier as the first parameter is WXg1, then the third parameter WX1=WXh1+WXg1. The Guest end can determine multiple third parameters.

[0080] The guest end determines the first gradient according to each third parameter, the second eigenvector corresponding to the third parameter, and the label value corresponding to the third parameter. The guest end can be regarded as completing a vertical federated learning when it obtains the first gradient. Specifically, the calculation of the first gradient has a corresponding first preset mapping relationship, and the guest end calculates the first gradient through the first preset mapping relationship, each third parameter, the second eigenvector corresponding to each third parameter, and the label value corresponding to each third parameter.

[0081] Step 203, determining an intermediate parameter according to the first gradient, and sending the intermediate parameter to the Host end, so that the Host end determines a second gradient of the Host end according to the intermediate parameter and each of the first eigenvectors;

[0082] In the process of calculating the first gradient, the guest end will first calculate the intermediate parameters, which are calculated by the label values ​​corresponding to each third parameter, and the guest end will calculate the first gradient through the intermediate parameters. In this regard, the guest end will determine the intermediate parameters according to the first gradient, and then send the intermediate parameters to the host end, so that the host end determines the second gradient of the host end according to the intermediate parameters and each of the first feature vectors, that is, the host end without a label can also calculate the second gradient of its own model.

[0083] Step 204: determining first sending information according to the first gradient and the first number of the second samples, and sending the first sending information to a third-party terminal, wherein the third-party terminal determines the first model parameter corresponding to the guest according to the first gradient and the first number sent by each guest, the second gradient sent by each host, and the second number of the first samples, so as to complete horizontal federated learning;

[0084] After the host and guest in the combination complete the vertical federated learning, it is necessary to integrate the gradients of all combinations for horizontal federated learning. The host generates a second transmission message based on the second gradient and the second number of the first sample, and sends the first transmission message to the third-party terminal; and the guest generates a second transmission message based on the calculated first gradient and the first number of the second sample, and sends the second transmission message to the third-party terminal. The third-party terminal can be any guest or server.

[0085] The third-party terminal parses the first transmission information and the second transmission information to obtain the first gradient and the first quantity of each Guest terminal, and obtains the second gradient and the second quantity of each Host terminal. The third-party terminal calculates the model parameters to be processed according to each first quantity, each first gradient, each second gradient and each second quantity. Specifically, the sum of each first quantity and each second quantity is obtained, and the first sum between each first gradient and each second gradient is obtained. The ratio of the first sum to the sum of the quantities is the model parameters to be processed. When each Guest terminal and each Host terminal send a transmission message to the third-party terminal, the transmission message carries its own device identification. The third-party terminal determines the model parameters of the terminal corresponding to the device identification based on the device identification, the number associated with the device identification and the model parameters to be processed. For example, if a Host terminal uses 10 first samples to calculate the gradient, the model parameters of the Host terminal are 10 times the model parameters to be processed.

[0086] The model parameters of the host or guest can refer to the following formula: in, is the model parameter, [[w]] is the sum of all first gradients and second gradients, n jk The number of samples involved in gradient calculation on the host or guest side.

[0087] It should be noted that there may be multiple first gradients in the Guest. For example, if the second sample has three features, such as height, weight, and age, the Guest calculates three gradients. The Guest determines a first combined feature vector based on each first gradient, and uses a first encryption algorithm to encrypt the first combined feature vector to obtain [W k ], then based on the encrypted first combined feature vector [W k ] and the number of second samples n jk Generate the first sending information. Similarly, the Host determines the second combined feature vector based on each second gradient, and uses the second encryption algorithm to encrypt the second combined feature vector to obtain [W k ], then based on the encrypted second combined feature vector [W k] and the number of first samples n jk Generate the second sending information. It can be understood that [[w]] is all [W k ] and.

[0088] After the third-party terminal obtains the first model parameter corresponding to each Guest terminal, it sends the first model parameter to the Guest terminal corresponding to the first model parameter. And after the third-party terminal obtains the second model parameter corresponding to each Host terminal, it sends the second model parameter to the Host terminal corresponding to the second model parameter. When the third-party terminal feeds back the model parameters to the Host terminal and the Guest terminal, it can be regarded as completing a horizontal federated learning.

[0089] Step 205: Receive the first model parameters fed back by the third-party terminal, update the parameters of the first model according to the first model parameters, and train the first model with updated parameters according to each of the second samples to obtain a first prediction model.

[0090] After receiving the first model parameters, the Guest end decrypts the first model parameters, thereby updating the parameters of the first model based on the decrypted first model parameters, and then uses each second sample to train the first model with updated parameters to obtain the first prediction model. Similarly, after receiving the second model parameters, the Host end decrypts the second model parameters, thereby updating the parameters of the second model based on the decrypted second model parameters, and then uses the first sample to train the second model with updated parameters to obtain the second prediction model.

[0091] It should be noted that after obtaining the model parameters to be processed, the third-party terminal needs to determine whether the Guest and Host meet the iteration conditions. If the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue the iteration information at the same time, that is, the first model parameters received by the Guest carry the information of continuing iteration. At this time, the Guest needs to continue to iterate to obtain the first gradient. The Guest re-determines the second sample in each sample in the Guest, and sets the weight of the second feature vector corresponding to the re-determined second sample according to the first model parameter, and then returns to execute the step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, so as to calculate the first gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the first model parameters received by the Guest do not carry the information of continuing iteration, and the first prediction model is obtained by training the first model of updating the parameters according to each second sample.

[0092] Similarly, if the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue iteration information at the same time, that is, the second model parameters received by the Host carry the information of continuing iteration. At this time, the Host needs to continue iterating to obtain the second gradient. The Host re-determines the first sample in each sample in the Host, and sets the weight of the first feature vector corresponding to the re-determined first sample according to the second model parameter, and then returns to the step of determining the first parameter, so as to calculate the second gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the second model parameters received by the Host do not carry the information of continuing iteration, and the second prediction model is obtained by training the second model with updated parameters according to each first sample.

[0093] It should be noted that the iteration conditions include: the number of iterations of the Guest and Host is less than the preset number, where the Guest and Host are iterated synchronously; the change between the last determined model parameters to be determined and the current determined model parameters to be determined is greater than the preset change. The weight update can be the current gradient minus the gradient of the model parameters fed back by the third-party terminal. For example, the weight of the feature vector of the sample pair in the Guest is W g1 , the gradient of the first model parameter is G g1 , then the updated weight W g2 =W g1 -G g1 For example, the weight of the feature vector of the sample pair in the Host is W h1 , the gradient of the first model parameter is G h1 , then the updated weight W h2 =W h1 -G h1 .

[0094] In addition, the number of second samples reselected by the Guest end may be any number, and the number of first samples reselected by the Host end may be any number.

[0095] After obtaining the first prediction model, the Guest can perform risk prediction for the user. Specifically, the Guest obtains a sample of the first user to be predicted, extracts a third feature vector from the sample, and inputs the third feature vector into the first prediction model. The Guest obtains the first numerical value output by the first prediction model, and the Guest can determine the prediction result of the first user based on the first numerical value. The specific description of the prediction result, the first numerical value, etc. refers to the above description, which will not be repeated here.

[0096] In the technical solution provided in this embodiment, the Guest and Host ends in the combination complete vertical federated learning through data interaction to obtain the gradient required by the model. The Guest and Host ends in the combination then send their respective gradients and the number of samples involved in calculating the gradients to the third-party terminal, so that the third-party terminal performs horizontal federated learning based on the gradients and the number of samples of each combination, and feeds back the model parameters obtained by the horizontal federated learning to the terminal corresponding to the model parameters, so that the Guest and Host ends update the parameters of the model based on the federated model parameters, and then use the samples to train the model with updated parameters to obtain a prediction model, that is, combining horizontal federated learning and vertical federated learning to obtain a prediction model with high accuracy of prediction results.

[0097] In one embodiment, determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector in step 201 includes:

[0098] Determine a second feature vector of a second sample corresponding to the identifier;

[0099] Determine a first parameter to be processed according to the second eigenvector and a weight corresponding to the second eigenvector;

[0100] The first parameter to be processed is encrypted using a first encryption algorithm to obtain a second parameter, wherein the second encryption algorithm used by the first parameter is a homomorphic encryption algorithm as the first encryption algorithm.

[0101] In this embodiment, the second sample in the Guest has an identifier, and the received first parameter carries the identifier. The Guest determines the second feature vector of the second sample based on the received identifier, and then multiplies the second feature vector and the weight corresponding to the second feature vector to obtain the first parameter to be processed WXg1, and the Guest then uses the first encryption algorithm to encrypt the first parameter to be processed to obtain the second parameter [WXg1].

[0102] It should be noted that after the Host obtains the second parameter to be processed WXh1 through the first eigenvector and the first eigenvector, the second encryption algorithm is used to obtain the first parameter [WXh1] for the second parameter to be processed WXh1. That is, the first parameter sent by the Host to the Guest is encrypted. The first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms, and are additively homomorphic. The third parameter = [WXg1] + [WXh1], so that the third parameter has a complete ciphertext, and then according to the complete ciphertext, that is, the Guest can use the encrypted first parameter and the second parameter to obtain the encrypted third parameter, thereby avoiding data leakage on the Host side.

[0103] In the technical solution provided in this embodiment, the Guest determines the second eigenvector of the second sample corresponding to the identifier, determines the first parameter to be processed according to the second eigenvector and the weight corresponding to the second eigenvector, and uses the first encryption algorithm to encrypt the first parameter to be processed to obtain the second parameter, so that the second parameter and the encrypted first parameter sent by the Host are calculated in an encrypted state to obtain the third parameter. Under the premise of avoiding data leakage on the Host side, the Guest side obtains the third parameter truly and accurately, and then accurately updates the parameters of the model, that is, under the premise of ensuring data security, the prediction accuracy of the first prediction model is improved.

[0104] In one embodiment, the step of determining the first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter in step 202 includes:

[0105] Determine a second sample corresponding to the third parameter;

[0106] Determine the label value of the second sample corresponding to the third parameter as the label value corresponding to the third parameter;

[0107] Determine a fourth parameter corresponding to the third parameter according to the third parameter, the correspondence of the third parameter and the label value, and determine the second feature vector of the second sample corresponding to the third parameter as the second feature vector corresponding to the fourth parameter;

[0108] A fifth parameter is determined according to the third quantity of the third parameter and each of the fourth parameters, and a first gradient is determined according to the fifth parameter and a second eigenvector corresponding to each of the fourth parameters.

[0109] In this embodiment, the Guest has a calculation formula for the first gradient. The calculation formula may be: gradient = (1 / N) * ∑ (0.5 * ywx-1) * 0.5yx = (1 / N) * ∑ (0.25 * wx-0.5 * y) * x, where gradient is the gradient, y is the label value, wx is the third parameter, and N is the third number of the third parameter. N can also be the number of second samples. gradient = (1 / N) * ∑ (0.5 * ywx-1) * 0.5yx = (1 / N) * ∑ (0.25 * wx-0.5 * y) * x can be transformed into: gradient = ∑ [[d]] x, where [[d]] is the fifth parameter, [[d]] = (1 / N) * ∑ (0.5 * ywx-1) * 0.5y, and x is the second eigenvector.

[0110] Specifically, the Guest determines the second sample corresponding to the third parameter, and the second sample is the sample participating in the calculation of the third parameter. Each second sample has a corresponding label value y, and the Guest determines the label value of the second sample corresponding to the third parameter as the label value corresponding to the third parameter. The Guest then determines the fourth parameter corresponding to the third parameter based on the third parameter and the label value corresponding to the third parameter. The fourth parameter is (0.5*ywx-1)*0.5y. The Guest then determines the fifth parameter based on the third quantity of the third parameter and each fourth eigenvector, and the fifth parameter is [[d]]=(1 / N)*∑(0.5*ywx-1)*0.5y. Finally, the Guest calculates the first gradient based on the fifth parameter and the second eigenvectors corresponding to each fourth parameter, that is, the first gradient gradient=∑[[d]]x.

[0111] The host does not have the label value y, and [[d]] includes all label values ​​y. The guest can send the fifth parameter to the host as an intermediate parameter, so that the host can calculate the second gradient through the intermediate parameter and the first eigenvector without the label value y, that is, the host uses gradient = ∑ [[d]]x to calculate the second gradient. It can be understood that the guest obtains the fifth parameter according to the first gradient and determines the fifth parameter as the intermediate parameter.

[0112] The calculation formula of the first gradient can be represented as a first preset mapping relationship, that is, the guest determines the first gradient according to each third parameter, the second feature vector corresponding to the third parameter, the label value corresponding to the third parameter, and the first preset mapping relationship. The first preset mapping relationship is determined according to the logistic regression algorithm. The following describes in detail how to obtain the calculation formula of the gradient according to the logistic regression algorithm.

[0113] The logistic regression algorithm has a corresponding logistic regression formula, which is: Among them, X is the feature vector, that is, the variable vector of the sample, such as height, etc.; W is the weight vector of X, referred to as the weight of X; and This type of function is called a Sigmoid function; y is the label value of the sample, and y is -1 or 1. Therefore, for a set of variable vectors, the probability of the predicted result being 1 is the result of the dot multiplication of this set of variables and their weights, and then the result obtained by the sigmoid function.

[0114] Conversely, the probability formula for the result prediction to be 0 is: 1-hθ(x).

[0115] The final unified probability function is: P(y|x;θ)=(h θ (x) y*(1-hθ(x)) 1-y When y is 0, its probability function is equal to 1-hθ(x) mentioned above, and when y is 1, its probability is hθ(x).

[0116] For P(y|x;θ)=(h θ (x) y *(1-hθ(x)) 1-y By finding log, we can get the Loss function of the logistic regression algorithm, and then take the derivative to get the gradient. The formula corresponding to the gradient is:

[0117]

[0118] Among them, the last term αW is a high-order infinitesimal and can be ignored. From this, the gradient formula can be derived as follows: gradient = (1 / N) * ∑ (0.5 * ywx-1) * 0.5yx = (1 / N) * ∑ (0.25 * wx-0.5 * y) * x.

[0119] Furthermore, after removing αW, the above gradient calculation formula is split into two parts, d and x, thereby deducing the intermediate parameter [[d]] = (1 / N)*∑(0.5*ywx-1)*0.5y.

[0120] In the technical solution provided in this embodiment, the Guest end determines the fourth parameter corresponding to the third parameter based on the third parameter, the second eigenvector corresponding to the third parameter, and the label value, and then determines the fifth parameter according to the third quantity of the third parameter and each fourth parameter. Finally, the first gradient is determined according to the fifth parameter and the second eigenvector corresponding to each fourth parameter, and the fifth parameter is determined as the intermediate parameter, so that the Guest end and the Host end can accurately determine their respective gradients.

[0121] Reference Figure 3 , Figure 3 Another flow chart of the model training method of vertical federated learning and horizontal federated learning of the present invention is applied to the Host side, and the method includes:

[0122] Step 301, determining each first sample from each sample in the Host end, and determining a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector;

[0123] In this embodiment, several guests and several hosts participate in the training of the prediction model. The number of guests is the same as the number of hosts, and a guest is combined with a corresponding host to perform vertical federated learning first. For example, if there are three guests and three hosts, there are three combinations, one combination consists of one guest and one host, and the three combinations perform federated vertical learning at the same time.

[0124] Each guest terminal holds data samples with the same characteristics and labels with the same meaning; each host terminal holds data samples with the same characteristics.

[0125] When performing vertical federated learning, the Host end in the combination selects multiple first samples from the data samples, and extracts the first feature vector Xh corresponding to the first samples, and each first sample has a corresponding identifier, which can be the id; the Guest end in the combination selects multiple second samples from the data samples, and extracts the second feature vector Xg corresponding to the second samples, and each second sample has a corresponding identifier, which can be the id, and at the same time, each second sample corresponds to a label value y, which can be -1 or 1.

[0126] Step 302: determining a first parameter according to the first feature vector and a weight corresponding to the first feature vector, and sending each of the first parameters and an identifier of a first sample corresponding to the first parameter to a guest;

[0127] The sample corresponds to multiple features. For example, the features can be age, gender, and height, that is, a sample can extract multiple features, and each feature is represented by a feature vector. The features are set with corresponding weights, that is, the feature vectors are set with corresponding weights. After extracting the first feature vector corresponding to the first sample, the host side obtains the first parameter according to the first feature vector Xh and the weight Wh corresponding to the first feature vector, that is, Xh and Wh are combined to obtain the first parameter WXh. The host determines the identifier of the first sample corresponding to the first feature vector, and associates the identifier with the first parameter. For example, the serial number of the first sample is 1, and the identifier is defined as N1, then the first parameter WXh is associated with N1. The host side can calculate multiple first parameters, and then send each first parameter and the identifier associated with the first parameter to the guest side.

[0128] The Guest end is also provided with weights corresponding to the feature vectors. The Guest end selects multiple second samples from its own data samples, and extracts the second feature vector Wg corresponding to each second sample, and the second sample corresponds to multiple second feature vectors Wg. The Guest end obtains the second parameter based on the second feature vector Xg and the weight Wg corresponding to the second feature vector, that is, Xg and Wg are combined to obtain the second parameter WXg. In the process of calculating the second parameter, the Guest can receive the first parameters sent by the Host end and the identifiers associated with the first parameters.

[0129] Step 303, receiving the intermediate parameters fed back by the guest, and determining a second gradient according to the intermediate parameters and each of the first feature vectors to complete the vertical federated learning;

[0130] The second parameter is determined by the second feature vector corresponding to the second sample, and the identifier corresponding to the second sample can be regarded as the identifier corresponding to the second parameter. The Guest end determines the first parameter and the second parameter with the same identifier from the received identifier, and thus obtains the third parameter based on the first parameter and the second parameter. For example, the first parameter is WXh1, and the second parameter with the same identifier as the first parameter is WXg1, then the third parameter WX1=WXh1+WXg1. The Guest end can determine multiple third parameters.

[0131] The guest end determines the first gradient according to each third parameter, the second eigenvector corresponding to the third parameter, and the label value corresponding to the third parameter. The guest end can be regarded as completing a vertical federated learning when it obtains the first gradient. Specifically, the calculation of the first gradient has a corresponding first preset mapping relationship, and the guest end calculates the first gradient through the first preset mapping relationship, each third parameter, the second eigenvector corresponding to each third parameter, and the label value corresponding to each third parameter.

[0132] In the process of calculating the first gradient, the guest end will first calculate the intermediate parameters, which are calculated by the label values ​​corresponding to each third parameter, and the guest end will calculate the first gradient through the intermediate parameters. In this regard, the guest end will determine the intermediate parameters according to the first gradient, and then send the intermediate parameters to the host end, so that the host end determines the second gradient of the host end according to the intermediate parameters and each of the first feature vectors, that is, the host end without a label can also calculate the second gradient of its own model.

[0133] Step 304: Generate second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals, so as to complete horizontal federated learning;

[0134] After the host and guest in the combination complete the vertical federated learning, it is necessary to integrate the gradients of all combinations for horizontal federated learning. The host generates a second transmission message based on the second gradient and the second number of the first sample, and sends the first transmission message to the third-party terminal; and the guest generates a second transmission message based on the calculated first gradient and the first number of the second sample, and sends the second transmission message to the third-party terminal. The third-party terminal can be any guest or server.

[0135] The third-party terminal parses the first transmission information and the second transmission information to obtain the first gradient and the first quantity of each Guest terminal, and obtains the second gradient and the second quantity of each Host terminal. The third-party terminal calculates the model parameters to be processed according to each first quantity, each first gradient, each second gradient and each second quantity. Specifically, the sum of each first quantity and each second quantity is obtained, and the first sum between each first gradient and each second gradient is obtained. The ratio of the first sum to the sum of the quantities is the model parameters to be processed. When each Guest terminal and each Host terminal send a transmission message to the third-party terminal, the transmission message carries its own device identification. The third-party terminal determines the model parameters of the terminal corresponding to the device identification based on the device identification, the number associated with the device identification and the model parameters to be processed. For example, if a Host terminal uses 10 first samples to calculate the gradient, the model parameters of the Host terminal are 10 times the model parameters to be processed.

[0136] The model parameters of the host or guest can refer to the following formula: in, is the model parameter, [[w]] is the sum of all first gradients and second gradients, n jk The number of samples involved in gradient calculation on the host or guest side.

[0137] It should be noted that there may be multiple first gradients in the Guest. For example, if the second sample has three features, such as height, weight, and age, the Guest calculates three gradients. The Guest determines a first combined feature vector based on each first gradient, and uses a first encryption algorithm to encrypt the first combined feature vector to obtain [W k ], then based on the encrypted first combined feature vector [Wk ] and the number of second samples n jk Generate the first sending information. Similarly, the Host determines the second combined feature vector based on each second gradient, and uses the second encryption algorithm to encrypt the second combined feature vector to obtain [W k ], then based on the encrypted second combined feature vector [W k ] and the number of first samples n jk Generate the second sending information. It can be understood that [[w]] is all [W k ] and.

[0138] After the third-party terminal obtains the first model parameter corresponding to each Guest terminal, it sends the first model parameter to the Guest terminal corresponding to the first model parameter. And after the third-party terminal obtains the second model parameter corresponding to each Host terminal, it sends the second model parameter to the Host terminal corresponding to the second model parameter. When the third-party terminal feeds back the model parameters to the Host terminal and the Guest terminal, it can be regarded as completing a horizontal federated learning.

[0139] Step 305: Receive the second model parameters fed back by the third terminal, update the parameters of the second model according to the second model parameters, and train the second model with updated parameters according to each of the first samples to obtain a second prediction model.

[0140] After receiving the second model parameters, the host end decrypts the second model parameters, thereby updating the parameters of the second model based on the decrypted second model parameters, and then uses the first sample to train the second model with updated parameters to obtain the second prediction model. Similarly, after receiving the first model parameters, the guest end decrypts the first model parameters, thereby updating the parameters of the first model based on the decrypted first model parameters, and then uses each second sample to train the first model with updated parameters to obtain the first prediction model.

[0141] It should be noted that after obtaining the model parameters to be processed, the third-party terminal needs to determine whether the Guest and Host meet the iteration conditions. If the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue the iteration information at the same time, that is, the first model parameters received by the Guest carry the information of continuing iteration. At this time, the Guest needs to continue to iterate to obtain the first gradient. The Guest re-determines the second sample in each sample in the Guest, and sets the weight of the second feature vector corresponding to the re-determined second sample according to the first model parameter, and then returns to execute the step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, so as to calculate the first gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the first model parameters received by the Guest do not carry the information of continuing iteration, and the first prediction model is obtained by training the first model of updating the parameters according to each second sample.

[0142] Similarly, if the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue iteration information at the same time, that is, the second model parameters received by the Host carry the information of continuing iteration. At this time, the Host needs to continue iterating to obtain the second gradient. The Host re-determines the first sample in each sample in the Host, and sets the weight of the first feature vector corresponding to the re-determined first sample according to the second model parameter, and then returns to the step of determining the first parameter, so as to calculate the second gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the second model parameters received by the Host do not carry the information of continuing iteration, and the second prediction model is obtained by training the second model with updated parameters according to each first sample.

[0143] It should be noted that the iteration conditions include: the number of iterations of the Guest and Host is less than the preset number, where the Guest and Host are iterated synchronously; the change between the last determined model parameters to be determined and the current determined model parameters to be determined is greater than the preset change. The weight update can be the current gradient minus the gradient of the model parameters fed back by the third-party terminal. For example, the weight of the feature vector of the sample pair in the Guest is W g1 , the gradient of the first model parameter is G g1 , then the updated weight W g2 =W g1 -G g1 For example, the weight of the feature vector of the sample pair in the Host is Wh1 , the gradient of the first model parameter is G h1 , then the updated weight W h2 =W h1 -G h1 .

[0144] In addition, the number of second samples reselected by the Guest end may be any number, and the number of first samples reselected by the Host end may be any number.

[0145] After obtaining the second prediction model, the Host can perform risk prediction for the user. Specifically, the Host obtains a sample of the second user to be predicted, extracts the fourth eigenvector from the sample, and inputs the fourth eigenvector into the second prediction model. The Host obtains the second numerical value output by the second prediction model, and the Host can determine the prediction result of the second user based on the second numerical value. The specific description of the prediction result, the second numerical value, etc. refers to the above description, which will not be repeated here.

[0146] In the technical solution provided in this embodiment, the Guest and Host ends in the combination complete vertical federated learning through data interaction to obtain the gradient required by the model. The Guest and Host ends in the combination then send their respective gradients and the number of samples involved in calculating the gradients to the third-party terminal, so that the third-party terminal performs horizontal federated learning based on the gradients and the number of samples of each combination, and feeds back the model parameters obtained by the horizontal federated learning to the terminal corresponding to the model parameters, so that the Guest and Host ends update the parameters of the model based on the federated model parameters, and then use the samples to train the model with updated parameters to obtain a prediction model, that is, combining horizontal federated learning and vertical federated learning to obtain a prediction model with high accuracy of prediction results.

[0147] In one embodiment, determining the first parameter according to the first feature vector and the weight corresponding to the first feature vector in step 302 includes:

[0148] Determine a second parameter to be processed according to the first feature vector and a weight corresponding to the first feature vector;

[0149] The second parameter to be processed is encrypted using a second encryption method to obtain a first parameter, wherein a first encryption algorithm used by the second parameter and the first encryption algorithm are homomorphic encryption algorithms, and the second parameter is used to determine the intermediate parameter.

[0150] The second sample in the guest end has an identifier, and the received first parameter carries the identifier. The guest end determines the second feature vector of the second sample based on the received identifier, and then multiplies the second feature vector and the weight corresponding to the second feature vector to obtain the first parameter to be processed WXg1. The guest end then uses the first encryption algorithm to encrypt the first parameter to be processed to obtain the second parameter [WXg1].

[0151] After the host obtains the second parameter to be processed WXh1 through the first feature vector and the first feature vector, the second encryption algorithm is used to obtain the first parameter [WXh1] for the second parameter to be processed WXh1. That is, the first parameter sent by the host to the guest is encrypted. The first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms, and are additively homomorphic. The third parameter = [WXg1] + [WXh1], so that the third parameter has a complete ciphertext, and then according to the complete ciphertext, that is, the guest can use the encrypted first parameter and the second parameter to obtain the encrypted third parameter, avoiding data leakage on the host side.

[0152] In the technical solution provided in this embodiment, the encryption algorithm used for the first parameter and the second parameter is a homomorphic encryption algorithm. Under the premise of avoiding data leakage on the host side, the guest side can obtain the third parameter truly and accurately, and then accurately update the parameters of the model. That is, under the premise of ensuring data security, the prediction accuracy of the first prediction model is improved.

[0153] In one embodiment, the calculation formula of the second gradient can be represented as a second preset mapping relationship, and the Host end determines the second gradient according to the second number of the first samples, the intermediate parameters, and each of the first feature vectors, and determines the first gradient according to the second preset mapping relationship, and the second preset mapping relationship is determined according to a logistic regression algorithm.

[0154] Specifically, the logistic regression algorithm has a corresponding logistic regression formula, which is: Among them, X is the feature vector, that is, the variable vector of the sample, such as height, etc.; W is the weight vector of X, referred to as the weight of X; and This type of function is called a Sigmoid function; y is the label value of the sample, and y is -1 or 1. Therefore, for a set of variable vectors, the probability of the predicted result being 1 is the result of the dot multiplication of this set of variables and their weights, and then the result obtained by the sigmoid function.

[0155] Conversely, the probability formula for the result prediction to be 0 is: 1-hθ(x).

[0156] The final unified probability function is: P(y|x;θ)=(h θ(x))y*(1-hθ(x)) 1 When y is 0, its probability function is equal to 1-hθ(x) mentioned above, and when y is 1, its probability is hθ(x).

[0157] For P(y|x;θ)=(h θ (x))y*(1-hθ(x)) 1 -y to find the log, we can get the Loss function of the logistic regression algorithm, and then take the derivative to get the gradient. The formula corresponding to the gradient is:

[0158]

[0159] Among them, the last term αW is a high-order infinitesimal and can be ignored. From this, the gradient formula can be derived as follows: gradient = (1 / N) * ∑ (0.5 * ywx-1) * 0.5yx = (1 / N) * ∑ (0.25 * wx-0.5 * y) * x.

[0160] Furthermore, after removing αW, the above gradient calculation formula is split into two parts, d and x, thereby deducing the intermediate parameter [[d]] = (1 / N)*∑(0.5*ywx-1)*0.5y.

[0161] Reference Figure 4 , Figure 4 This is another flow chart of the model training method of vertical federated learning and horizontal federated learning of the present invention, which is applied to a third-party terminal. The method includes:

[0162] Step 401, obtaining first transmission information sent by each guest end and second transmission information sent by each host end, and obtaining a first gradient and a first number of second samples according to the first transmission information, and obtaining a second gradient and a second number of first samples according to the second transmission information;

[0163] Step 402, determining a model parameter to be determined according to the first gradient, the first quantity, the second gradient, and the second quantity;

[0164] Step 403: generating first model parameters corresponding to the Guest end according to the model parameters to be processed and the first number corresponding to the Guest end, and generating second model parameters corresponding to the Host end according to the model parameters to be processed and the second number corresponding to the Host end;

[0165] Step 404: Send the first model parameter to the Guest end corresponding to the first model parameter, and send the second model parameter to the Host end corresponding to the second model parameter.

[0166] After the host and guest in the combination complete the vertical federated learning, it is necessary to integrate the gradients of all combinations for horizontal federated learning. The host generates a second transmission message based on the second gradient and the second number of the first sample, and sends the first transmission message to the third-party terminal; and the guest generates a second transmission message based on the calculated first gradient and the first number of the second sample, and sends the second transmission message to the third-party terminal. The third-party terminal can be any guest or server.

[0167] The third-party terminal parses the first transmission information and the second transmission information to obtain the first gradient and the first quantity of each Guest terminal, and obtains the second gradient and the second quantity of each Host terminal. The third-party terminal calculates the model parameters to be processed according to each first quantity, each first gradient, each second gradient and each second quantity. Specifically, the sum of each first quantity and each second quantity is obtained, and the first sum between each first gradient and each second gradient is obtained. The ratio of the first sum to the sum of the quantities is the model parameters to be processed. When each Guest terminal and each Host terminal send a transmission message to the third-party terminal, the transmission message carries its own device identification. The third-party terminal determines the model parameters of the terminal corresponding to the device identification based on the device identification, the number associated with the device identification and the model parameters to be processed. For example, if a Host terminal uses 10 first samples to calculate the gradient, the model parameters of the Host terminal are 10 times the model parameters to be processed.

[0168] The model parameters of the host or guest can refer to the following formula: in, is the model parameter, [[w]] is the sum of all first gradients and second gradients, n jk The number of samples involved in gradient calculation on the host or guest side.

[0169] It should be noted that there may be multiple first gradients in the Guest. For example, if the second sample has three features, such as height, weight, and age, the Guest calculates three gradients. The Guest determines a first combined feature vector based on each first gradient, and uses a first encryption algorithm to encrypt the first combined feature vector to obtain [W k ], then based on the encrypted first combined feature vector [W k ] and the number of second samples n jk Generate the first sending information. Similarly, the Host determines the second combined feature vector based on each second gradient, and uses the second encryption algorithm to encrypt the second combined feature vector to obtain [W k ], then based on the encrypted second combined feature vector [W k] and the number of first samples n jk Generate the second sending information. It can be understood that [[w]] is all [W k ] is the vector sum.

[0170] After the third-party terminal obtains the first model parameter corresponding to each Guest terminal, it sends the first model parameter to the Guest terminal corresponding to the first model parameter. And after the third-party terminal obtains the second model parameter corresponding to each Host terminal, it sends the second model parameter to the Host terminal corresponding to the second model parameter. When the third-party terminal feeds back the model parameters to the Host terminal and the Guest terminal, it can be regarded as completing a horizontal federated learning.

[0171] After obtaining the model parameters to be processed, the third-party terminal needs to determine whether the Guest and Host meet the iteration conditions. If the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue iteration information at the same time, that is, the first model parameters received by the Guest carry the information of continued iteration. At this time, the Guest needs to continue iterating to obtain the first gradient. The Guest re-determines the second sample in each sample in the Guest, and sets the weight of the second feature vector corresponding to the re-determined second sample according to the first model parameter, and then returns to execute the step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, so as to calculate the first gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the first model parameters received by the Guest do not carry the information of continued iteration, and the first prediction model is obtained by training the first model with updated parameters according to each second sample.

[0172] Similarly, if the Guest and Host meet the iteration conditions, the third-party terminal will feedback the model parameters and continue iteration information at the same time, that is, the second model parameters received by the Host carry the information of continuing iteration. At this time, the Host needs to continue iterating to obtain the second gradient. The Host re-determines the first sample in each sample in the Host, and sets the weight of the first feature vector corresponding to the re-determined first sample according to the second model parameter, and then returns to the step of determining the first parameter, so as to calculate the second gradient again. When the third-party terminal determines that the Guest and Host do not meet the iteration conditions, the third-party terminal only feeds back the model parameters, that is, the second model parameters received by the Host do not carry the information of continuing iteration, and the second prediction model is obtained by training the second model with updated parameters according to each first sample.

[0173] The Guest and Host in the combination complete vertical federated learning through data interaction to obtain the gradient required by the model. The Guest and Host in the combination then send their respective gradients and the number of samples involved in calculating the gradients to the third-party terminal, so that the third-party terminal performs horizontal federated learning based on the gradients and the number of samples of each combination, and feeds back the model parameters obtained by the horizontal federated learning to the terminal corresponding to the model parameters, so that the Guest and Host update the model parameters based on the federated model parameters, and then use the samples to train the model with updated parameters to obtain a prediction model, that is, combining horizontal federated learning and vertical federated learning to obtain a prediction model with high accuracy of prediction results.

[0174] Based on the above embodiment, taking two guest terminals and two host terminals as an example, the interaction of terminals involved in the model training method of vertical federated learning and horizontal federated learning provided by the present invention is briefly described. Figure 5 ,include:

[0175] S1a, Host1 calculate the first eigenvector x of the first sample h The weight corresponding to the first eigenvector is obtained as WX, and WX is encrypted to obtain [WX h ], and [WX h ] and identification ID are sent to Guest1.

[0176] S2a, Guest1 calculates the second eigenvector x of the second sample g The weight corresponding to the second eigenvector is obtained by encrypting WX to obtain [WX g ], and receive [WX h ] and identification ID.

[0177] S3a, Guest1 will have the same identifier [WX g ] and [WX h ] are superimposed to obtain [WX], Guest1 calculates [WX] based on each [WX], x g And the label value y of the second sample in Guest1 is used to calculate the first gradient G g , and based on the first gradient G g Obtain the intermediate parameter [[d]], Guest1 generates [Wk] according to the first gradient, and obtains the number of second samples n jk ;

[0178] S4a, Guest1 sends [[d]] to Host1;

[0179] S5a, Guest1 will [Wk] and the number of the second sample n jk Send to a third-party terminal;

[0180] S6a, Host1 according to [[d]] and each first eigenvector x h Get the second gradient G h , Host1 generates [Wk] according to the second gradient and obtains the number of first samples n jk ;

[0181] S7a, Host1 will [Wk] and the number of second samples n jk Send to a third-party terminal.

[0182] S1b, Host2 calculate the first eigenvector x of the first sample h The weight corresponding to the first eigenvector is obtained as WX, and WX is encrypted to obtain [WX h ], and [WX h ] and identification ID are sent to Guest2.

[0183] S2b, Guest2 calculate the second eigenvector x of the second sample g The weight corresponding to the second eigenvector is obtained by encrypting WX to obtain [WX g ], and receive [WX h ] and identification ID.

[0184] S3b, Guest2 will have the same identifier [WX g ] and [WX h ] are superimposed to obtain [WX], and Guest2 calculates [WX] based on each [WX], x g And the label value y of the second sample in Guest2 is calculated to obtain the first gradient G g , and based on the first gradient G g Obtain the intermediate parameter [[d]], Guest1 generates [Wk] according to the first gradient, and obtains the number of second samples n jk ;

[0185] S4b, Guest2 sends [[d]] to Host2;

[0186] S5b, Guest 2 will [Wk] and the number of second samples n jk Send to a third-party terminal;

[0187] S6b, Host1 according to [[d]] and each first eigenvector x h Get the second gradient G h , Host2 generates [Wk] according to the second gradient and obtains the number of first samples n jk ;

[0188] S7b, Host2 will [Wk] and the number of second samples n jk Send to a third-party terminal.

[0189] Step 1a to step 7a and step 1b to step 7b are performed simultaneously.

[0190] S8, the third-party terminal receives [Wk] and n sent by Host1, Host2, Guest1 and Guest2 jk Determine the total model parameters W, Then according to W and n sent by the terminal jk Get the model parameters corresponding to the terminal

[0191] S9, when the Host and Guest do not meet the iteration conditions, the third-party terminal will (w h1 ) is sent to Host1;

[0192] S10, Host1 according to (w h1 ) Update the parameters of the model, and use each x h The model with updated parameters is trained to obtain a prediction model.

[0193] S11, when the Host and Guest do not meet the iteration conditions, the third-party terminal will (w g1 ) is sent to Guest1;

[0194] S12, Guest1 (w g1 ) Update the parameters of the model, and use each x g The model with updated parameters is trained to obtain a prediction model.

[0195] S13, when the Host and Guest do not meet the iteration conditions, the third-party terminal will (w h2 ) is sent to Host2;

[0196] S14, Host2 according to (w h2 ) Update the parameters of the model, and use each x h The model with updated parameters is trained to obtain a prediction model.

[0197] S15, when the Host and Guest do not meet the iteration conditions, the third-party terminal will (w g2 ) is sent to Guest2;

[0198] S16, Guest2 (w g2 ) Update the parameters of the model, and use each x g The model with updated parameters is trained to obtain a prediction model.

[0199] The embodiment of the present invention also provides a prediction terminal. Figure 6 , Figure 6 A structural diagram of a prediction terminal provided in an embodiment of the present invention. Since the principle of solving the problem by the prediction terminal is similar to the model training method of vertical federated learning and horizontal federated learning in the embodiment of the present invention, the implementation of the prediction terminal and the technical effects achieved can refer to the above description, and the repeated parts will not be repeated.

[0200] like Figure 6 As shown, the prediction terminal 600 includes:

[0201] The first acquisition module 601 is used to obtain a target feature vector corresponding to a user's sample and input the target feature vector into a prediction model, wherein the prediction model is trained by using a plurality of first samples of the guest end and a plurality of second samples of the host end through a vertical federated learning method and a horizontal federated learning method;

[0202] The first acquisition module 601 is further used to acquire the numerical value output by the prediction model and determine the prediction result of the user according to the numerical value.

[0203] The embodiment of the present invention also provides a guest terminal. Figure 7 , Figure 7 A structural diagram of the Guest end provided for an embodiment of the present invention. Since the principle of solving the problem by the Guest end is similar to the model training method of vertical federated learning and horizontal federated learning in the embodiment of the present invention, the implementation of the Guest end and the technical effects achieved can refer to the above description, and the repetitions will not be repeated.

[0204] like Figure 7 As shown, the Guest terminal 700 includes:

[0205] The first receiving module 701 is used to receive each first parameter sent by the Host end and an identifier corresponding to the first parameter, and determine the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the Host end and the weight corresponding to the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter;

[0206] A first determination module 702 is used to determine a third parameter based on the first parameter and the second parameter of the same identifier, and determine a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter, so as to complete vertical federated learning;

[0207] The first determination module 701 is further configured to determine an intermediate parameter according to the first gradient, and send the intermediate parameter to the host end, so that the host end determines a second gradient of the host end according to the intermediate parameter and each of the first eigenvectors;

[0208] A first sending module 703 is used to determine first sending information according to the first gradient and the first number of the second samples, and send the first sending information to a third-party terminal, wherein the third-party terminal determines the first model parameter corresponding to the guest according to the first gradient and the first number sent by each guest, the second gradient sent by each host, and the second number of the first samples, so as to complete horizontal federated learning;

[0209] The first receiving module 702 is further used to receive the first model parameters fed back by the third-party terminal, update the parameters of the first model according to the first model parameters, and train the first model with updated parameters according to each of the second samples to obtain a first prediction model.

[0210] In one embodiment, the prediction terminal 700 includes:

[0211] A first determining module 702, configured to determine a second feature vector of a second sample corresponding to the identifier;

[0212] Determine a first parameter to be processed according to the second eigenvector and a weight corresponding to the second eigenvector;

[0213] An encryption module is used to encrypt the first parameter to be processed by using a first encryption algorithm to obtain a second parameter, wherein the second encryption algorithm used by the first parameter is a homomorphic encryption algorithm with the first encryption algorithm.

[0214] In one embodiment, the prediction terminal 700 includes:

[0215] A first determining module 702, configured to determine a second sample corresponding to the third parameter;

[0216] A first determining module 702, configured to determine the label value of the second sample corresponding to the third parameter as the label value corresponding to the third parameter;

[0217] A first determining module 702, configured to determine a fourth parameter corresponding to the third parameter according to the third parameter, a second eigenvector corresponding to the third parameter, and a label value;

[0218] The first determination module 702 is configured to determine a fifth parameter according to a third quantity of the third parameter and each of the fourth parameters, and determine a first gradient according to a second eigenvector corresponding to the fifth parameter and each of the fourth parameters.

[0219] In one embodiment, the prediction terminal 700 includes:

[0220] an acquisition module, configured to acquire the fifth parameter according to the first gradient;

[0221] The first determining module 702 is configured to determine the fifth parameter as an intermediate parameter.

[0222] In one embodiment, the prediction terminal 700 includes:

[0223] A first determination module 702, used to determine whether the first model parameter carries information for continuing iteration;

[0224] An execution module, configured to execute the step of training the first model with updated parameters according to each of the second samples to obtain a first prediction model when the first model parameters do not carry the information for continuing iteration;

[0225] A first determination module 702 is used to re-determine a second sample from each sample in the Guest end when the first model parameter carries the information of continuing iteration, and set the weight of the second feature vector corresponding to the re-determined second sample according to the first model parameter, wherein the third-party terminal sends the information of continuing iteration to each of the Guest end and the Host end when determining that the Guest end and the Host end meet the iteration condition;

[0226] The execution module is used to return to the step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector when receiving each first parameter and the identifier corresponding to the first parameter sent by the Host.

[0227] In one embodiment, the prediction terminal 700 includes:

[0228] An acquisition module, used for acquiring a third feature vector corresponding to the sample of the first user, and inputting the third feature vector into the first prediction model;

[0229] An acquisition module is used to acquire a first value output by the first prediction model and determine a prediction result of the first user based on the first value.

[0230] In one embodiment, the prediction terminal 700 includes:

[0231] A first determining module 702, configured to determine a first combined feature vector according to each of the first gradients, and encrypt the first combined feature vector using a first encryption algorithm;

[0232] A generating module is used to generate first sending information according to the encrypted first combined feature vector and the first number of the second samples.

[0233] The embodiment of the present invention also provides a host end. Figure 8 , Figure 8 A structural diagram of the Host end provided for an embodiment of the present invention. Since the principle of solving the problem on the Host end is similar to the model training method of vertical federated learning and horizontal federated learning in the embodiment of the present invention, the implementation of the Host end and the technical effects achieved can refer to the above description, and the repetitions will not be repeated.

[0234] like Figure 8 As shown, the host end 800 includes:

[0235] A second determination module 801 is used to determine each first sample from each sample in the Host end, and determine a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector;

[0236] The second determination module 801 is used to determine the first parameter according to the first feature vector and the weight corresponding to the first feature vector, and send the first parameter and the identifier of the first sample corresponding to the first parameter to the guest end;

[0237] A second receiving module 802 is used to receive the intermediate parameters fed back by the guest, and determine a second gradient according to the intermediate parameters and each of the first feature vectors to complete the vertical federated learning;

[0238] A second sending module 803 is used to generate second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals to complete horizontal federated learning;

[0239] The second receiving module 802 is further used to receive the second model parameters fed back by the third terminal, update the parameters of the second model according to the second model parameters, and train the second model with updated parameters according to each of the first samples to obtain a second prediction model.

[0240] In one embodiment, the host end 800 includes:

[0241] The second determination module 801 is used to determine a second parameter to be processed according to the first feature vector and a weight corresponding to the first feature vector;

[0242] An encryption module is used to encrypt the second parameter to be processed by a second encryption method to obtain a first parameter, wherein the first encryption algorithm used by the second parameter is a homomorphic encryption algorithm with the first encryption algorithm, and the second parameter is used to determine the intermediate parameter.

[0243] In one embodiment, the host end 800 includes:

[0244] The second determination module 801 is used to determine whether the second model parameter carries the information of continuing iteration;

[0245] An execution module, configured to execute the step of training the second model with updated parameters according to each of the first samples to obtain a second prediction model when the second model parameters do not carry the information of continuing iteration;

[0246] The second determination module 801 is used to re-determine the first sample from each sample in the Host end when the second model parameter carries the continue iteration information, and set the weight of the first feature vector corresponding to the re-determined first sample according to the second model parameter, wherein the third-party terminal sends the continue iteration information to each of the Guest end and the Host end when determining that the Guest end and the Host end meet the iteration condition;

[0247] An execution module is used to return to execute the step of determining the first parameter according to the first feature vector and the weight corresponding to the first feature vector.

[0248] In one embodiment, the host end 800 includes:

[0249] an acquisition module, configured to acquire a fourth feature vector corresponding to the sample of the second user, and input the third feature vector into the second prediction model;

[0250] An acquisition module is used to acquire a second value output by the second prediction model and determine a prediction result for the second user based on the second value.

[0251] In one embodiment, the host end 800 includes:

[0252] The second determining module 801 is used to determine a second combined feature vector according to each of the second gradients, and encrypt the first combined feature vector using a second encryption algorithm;

[0253] A generating module is used to generate second sending information according to the encrypted second combined feature vector and the second number of the first samples.

[0254] The embodiment of the present invention also provides a third-party terminal. Fig. 9 , Fig. 9 A structural diagram of the Host end provided for an embodiment of the present invention. Since the principle of solving the problem by the third-party terminal is similar to the model training method of vertical federated learning and horizontal federated learning in the embodiment of the present invention, the implementation of the third-party terminal and the technical effects achieved can refer to the above description, and the repetitions will not be repeated.

[0255] like Fig. 9 As shown, the third-party terminal 900 includes:

[0256] A second acquisition module 901 is used to acquire first transmission information sent by each guest end and second transmission information sent by each host end, and acquire a first gradient and a first number of second samples according to the first transmission information, and acquire a second gradient and a second number of first samples according to the second transmission information;

[0257] A third determining module 902, configured to determine a model parameter to be determined according to the first gradient, the first quantity, the second gradient, and the second quantity;

[0258] A generating module 903, configured to generate a first model parameter corresponding to the guest end according to the model parameter to be processed and the first quantity corresponding to the guest end, and to generate a second model parameter corresponding to the host end according to the model parameter to be processed and the second quantity corresponding to the host end;

[0259] The third sending module 904 is used to send the first model parameter to the Guest end corresponding to the first model parameter, and send the second model parameter to the Host end corresponding to the second model parameter.

[0260] In one embodiment, the third-party terminal 900 includes:

[0261] The third determination module 902 is used to determine whether the guest end and the host end meet the iteration condition;

[0262] The third sending module 904 is used to send the first model parameter and the continue iteration information to the Guest end corresponding to the first model parameter when the iteration condition is met at the Guest end and the Host end, and send the second model parameter and the continue iteration information to the Host end corresponding to the second model parameter.

[0263] In one embodiment, the third-party terminal 900 includes:

[0264] A third determination module 902 is used to determine the vector sum between each first combination feature vector and each second combination feature vector, wherein the first sent information includes the first combination feature vector, the first combination feature vector is determined according to each first gradient of the guest end, and the second sent information includes the second combination feature vector, and the second combination feature vector is determined according to each second gradient of the host end;

[0265] A third determining module 902 is used to determine the sum of each of the first quantities and each of the second quantities;

[0266] The third determination module 902 is used to determine the model parameters to be determined according to the vector sum and the quantity sum.

[0267] In one embodiment, if Fig.10 As shown, an embodiment of the present invention further provides an electronic device 1000, including a processor 1001, a memory 1002, and a program or instruction stored in the memory 1002 and executable on the processor 1001. When the program or instruction is executed by the processor 1001, each process of the above-mentioned method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0268] The embodiment of the present invention further provides a computer-readable storage medium, wherein a program or instruction is stored on the computer-readable storage medium, and when the program or instruction is executed by a processor, the above Figures 1 to 4 The various processes of any method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0269] The embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned Figures 1 to 4 The various processes of any method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0270] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0271] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0272] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0273] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A model training method for vertical federated learning and horizontal federated learning, applied to the guest, characterized in that: The model training method of the vertical federated learning and the horizontal federated learning includes: The Guest end receives each first parameter sent by the Host end and an identifier corresponding to the first parameter, and determines the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the Host end and the weight corresponding to the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter; wherein the Guest end is a terminal of a bank having samples and labels corresponding to the samples, and the Host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is characterized by a feature vector; Determine a third parameter based on the first parameter and the second parameter of the same identifier, and determine a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter, so as to complete vertical federated learning; determining an intermediate parameter according to the first gradient, and sending the intermediate parameter to the host end, so that the host end determines a second gradient of the host end according to the intermediate parameter and each of the first eigenvectors; Determine a first sending information according to the first gradient and the first number of the second samples, and send the first sending information to a third-party terminal, wherein the third-party terminal determines the first model parameter corresponding to the Guest according to the first gradient sent by each Guest, the first number, the second gradient sent by each Host, and the second number of the first samples, so as to complete horizontal federated learning; wherein the Guest determines a first combined feature vector based on each first gradient, encrypts the first combined feature vector using a first encryption algorithm, generates the first sending information based on the encrypted first combined feature vector and the first number of the second samples, the Host determines a second combined feature vector based on each second gradient, encrypts the second combined feature vector using a second encryption algorithm, generates the second sending information based on the encrypted second combined feature vector and the second number of the first samples, and sends the second sending information to the third-party terminal, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; receiving the first model parameters fed back by the third-party terminal, updating the parameters of the first model according to the first model parameters, and training the first model with updated parameters according to each of the second samples to obtain a first prediction model; wherein, after receiving the first model parameters, decrypting the first model parameters, and updating the parameters of the first model according to the decrypted first model parameters; Among them, after obtaining the first prediction model, the Guest side performs risk prediction on the user, wherein a sample of the first user to be predicted is obtained, and a third feature vector is extracted from the sample of the first user to be predicted, and the third feature vector is input into the first prediction model, and the first numerical value is output, and the prediction result of the first user is determined according to the first numerical value.

2. The model training method for vertical federated learning and horizontal federated learning as claimed in claim 1, characterized in that: The step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector comprises: Determine a second feature vector of a second sample corresponding to the identifier; Determine a first parameter to be processed according to the second eigenvector and a weight corresponding to the second eigenvector; The first parameter to be processed is encrypted using a first encryption algorithm to obtain a second parameter.

3. The model training method for vertical federated learning and horizontal federated learning according to claim 1, characterized in that: The step of determining the first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and the label value corresponding to the third parameter comprises: Determine a second sample corresponding to the third parameter; Determine the label value of the second sample corresponding to the third parameter as the label value corresponding to the third parameter; Determine a fourth parameter corresponding to the third parameter according to the third parameter, a second eigenvector corresponding to the third parameter, and a label value; A fifth parameter is determined according to the third quantity of the third parameter and each of the fourth parameters, and a first gradient is determined according to the fifth parameter and a second eigenvector corresponding to each of the fourth parameters.

4. The model training method for vertical federated learning and horizontal federated learning as claimed in claim 3, characterized in that: The step of determining an intermediate parameter according to the first gradient comprises: Acquire the fifth parameter according to the first gradient; The fifth parameter is determined as an intermediate parameter.

5. The model training method for vertical federated learning and horizontal federated learning according to any one of claims 1 to 4, characterized in that: After the step of updating the parameters of the first model according to the first model parameters, the method further includes: Determining whether the first model parameter carries information for continuing iteration; When the first model parameters do not carry the information of continuing iteration, performing the step of training the first model with updated parameters according to each of the second samples to obtain the first prediction model; When the first model parameter carries the information of continuing iteration, a second sample is re-determined from each sample in the guest end, and a weight of a second feature vector corresponding to the re-determined second sample is set according to the first model parameter, wherein the third-party terminal sends the information of continuing iteration to each of the guest end and the host end when determining that the guest end and the host end meet the iteration condition; When receiving each first parameter sent by the Host and an identifier corresponding to the first parameter, return to execute the step of determining the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector.

6. The model training method for vertical federated learning and horizontal federated learning as described in any one of claims 1 to 4, characterized in that: The Guest determines a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, the label value corresponding to the third parameter, and a first preset mapping relationship, where the first preset mapping relationship is determined according to a logistic regression algorithm.

7. A model training method for vertical federated learning and horizontal federated learning, applied to the host side, characterized in that: The model training method of the vertical federated learning and the horizontal federated learning includes: Determine each first sample from each sample in the Host end, and determine a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector; Determine a first parameter according to the first feature vector and the weight corresponding to the first feature vector, and send each of the first parameters and the identifier of the first sample corresponding to the first parameter to the Guest end; wherein the Guest end is a terminal of a bank having samples and labels corresponding to the samples, and the Host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is represented by a feature vector; Receiving the intermediate parameters fed back by the guest, and determining a second gradient according to the intermediate parameters and each of the first feature vectors to complete the vertical federated learning; Generate a second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals to complete horizontal federated learning; wherein the Guest terminal determines a first combined feature vector based on each of the first gradients, encrypts the first combined feature vector using a first encryption algorithm, generates a first sending information based on the encrypted first combined feature vector and the first number of the second samples, and sends the first sending information to the third-party terminal, the Host terminal determines a second combined feature vector based on each of the second gradients, encrypts the second combined feature vector using a second encryption algorithm, and generates a second sending information based on the encrypted second combined feature vector and the second number of the first samples, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; receiving the second model parameters fed back by the third-party terminal, updating the parameters of the second model according to the second model parameters, and training the second model after the updated parameters according to each of the first samples to obtain a second prediction model; wherein, after receiving the second model parameters, decrypting the second model parameters, and updating the parameters of the second model according to the decrypted second model parameters; After obtaining the second prediction model, the Host performs risk prediction on the user, wherein a sample of the second user to be predicted is obtained, and a fourth eigenvector is extracted from the sample of the second user to be predicted, the fourth eigenvector is input into the second prediction model, a second numerical value is output, and a prediction result of the second user is determined based on the second numerical value.

8. The model training method for vertical federated learning and horizontal federated learning according to claim 7, characterized in that: The step of determining the first parameter according to the first feature vector and the weight corresponding to the first feature vector comprises: Determine a second parameter to be processed according to the first feature vector and a weight corresponding to the first feature vector; The second parameter to be processed is encrypted using a second encryption method to obtain a first parameter, wherein the second parameter is used to determine the intermediate parameter.

9. The model training method for vertical federated learning and horizontal federated learning according to claim 7, characterized in that: After the step of updating the parameters of the second model according to the second model parameters, the method further includes: determining whether the second model parameter carries information for continuing iteration; When the second model parameters do not carry the information of continuing iteration, performing the step of training the second model with updated parameters according to each of the first samples to obtain a second prediction model; When the second model parameter carries the information of continuing iteration, the first sample is re-determined from each sample in the Host end, and the weight of the first feature vector corresponding to the re-determined first sample is set according to the second model parameter, wherein the third-party terminal sends the information of continuing iteration to each of the Guest end and the Host end when determining that the Guest end and the Host end meet the iteration condition; Return to the step of determining the first parameter according to the first eigenvector and the weight corresponding to the first eigenvector.

10. The model training method for vertical federated learning and horizontal federated learning according to any one of claims 7 to 9, characterized in that: The Host determines a second gradient according to the second number of the first samples, the intermediate parameters, and each of the first feature vectors, and determines the first gradient according to a second preset mapping relationship, where the second preset mapping relationship is determined according to a logistic regression algorithm.

11. A model training method for vertical federated learning and horizontal federated learning, applied to a third-party terminal, characterized in that: The model training method of the vertical federated learning and the horizontal federated learning includes: Obtain the first transmission information sent by each Guest end and the second transmission information sent by each Host end, and obtain the first gradient and the first number of the second samples according to the first transmission information, and obtain the second gradient and the second number of the first samples according to the second transmission information; wherein the Guest end is a terminal of a bank having samples and labels corresponding to the samples, and the Host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is represented by a feature vector; wherein the Guest end determines a first combined feature vector based on each first gradient, and encrypts the first combined feature vector using a first encryption algorithm, generates a first transmission information based on the encrypted first combined feature vector and the first number of the second samples, and sends the first transmission information to a third-party terminal, the Host end determines a second combined feature vector based on each second gradient, and encrypts the second combined feature vector using a second encryption algorithm, generates a second transmission information based on the encrypted second combined feature vector and the second number of the first samples, and sends the second mode information to the third-party terminal, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; Determine a model parameter to be processed according to the first gradient, the first quantity, the second gradient, and the second quantity; Generate a first model parameter corresponding to the Guest end according to the model parameter to be processed and the first number corresponding to the Guest end, and generate a second model parameter corresponding to the Host end according to the model parameter to be processed and the second number corresponding to the Host end; The first model parameter is sent to a Guest end corresponding to the first model parameter, and the second model parameter is sent to a Host end corresponding to the second model parameter.

12. The model training method for vertical federated learning and horizontal federated learning according to claim 11, characterized in that: The step of sending the first model parameter to the Guest end corresponding to the first model parameter, and sending the second model parameter to the Host end corresponding to the second model parameter includes: Determine whether the guest end and the host end meet the iteration condition; When the guest end and the host end meet the iteration condition, the first model parameter and the continue iteration information are sent to the guest end corresponding to the first model parameter, and the second model parameter and the continue iteration information are sent to the host end corresponding to the second model parameter.

13. The model training method for vertical federated learning and horizontal federated learning according to claim 11 or 12, characterized in that: The step of determining the parameters of the model to be processed according to the first gradient, the first quantity, the second gradient and the second quantity comprises: Determine a vector sum between each first combination feature vector and each second combination feature vector, wherein the first sent information includes the first combination feature vector, the first combination feature vector is determined according to each first gradient of the guest end, and the second sent information includes the second combination feature vector, and the second combination feature vector is determined according to each second gradient of the host end; determining a sum of each of the first quantities and each of the second quantities; The model parameters to be processed are determined according to the vector sum and the quantity sum.

14. A guest terminal, characterized in that: The Guest includes: A first receiving module is used to receive each first parameter sent by the Host end and an identifier corresponding to the first parameter, and determine the second parameter according to the second feature vector of the second sample corresponding to the identifier and the corresponding weight of the second feature vector, wherein the first parameter is determined according to the first feature vector of the first sample in the Host end and the weight corresponding to the first feature vector, and the identifier is the identifier of the first sample corresponding to the first parameter; wherein the Guest end is a terminal of a bank having samples and labels corresponding to the samples, and the Host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is characterized by a feature vector; A first determination module, configured to determine a third parameter based on the first parameter and the second parameter of the same identifier, and determine a first gradient according to each of the third parameters, the second feature vector corresponding to the third parameter, and a label value corresponding to the third parameter, so as to complete vertical federated learning; The first determination module is further configured to determine an intermediate parameter according to the first gradient, and send the intermediate parameter to the Host end, so that the Host end determines a second gradient of the Host end according to the intermediate parameter and each of the first eigenvectors; A first sending module, used to determine first sending information according to the first gradient and the first number of the second samples, and send the first sending information to a third-party terminal, wherein the third-party terminal determines the first model parameter corresponding to the Guest according to the first gradient sent by each Guest, the first number, the second gradient sent by each Host, and the second number of the first samples, so as to complete horizontal federated learning; wherein the Guest determines a first combined feature vector based on each first gradient, and encrypts the first combined feature vector using a first encryption algorithm, and generates the first sending information based on the encrypted first combined feature vector and the first number of the second samples, the Host determines a second combined feature vector based on each second gradient, and encrypts the second combined feature vector using a second encryption algorithm, and generates the second sending information based on the encrypted second combined feature vector and the second number of the first samples, and sends the second sending information to the third-party terminal, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; The first receiving module is further used to receive the first model parameters fed back by the third-party terminal, and update the parameters of the first model according to the first model parameters, and train the first model with updated parameters according to each of the second samples to obtain a first prediction model; wherein, after receiving the first model parameters, the first model parameters are decrypted, and the parameters of the first model are updated according to the decrypted first model parameters; Among them, after obtaining the first prediction model, the Guest side performs risk prediction on the user, wherein a sample of the first user to be predicted is obtained, and a third feature vector is extracted from the sample of the first user to be predicted, and the third feature vector is input into the first prediction model, and the first numerical value is output, and the prediction result of the first user is determined according to the first numerical value.

15. A host end, characterized in that: The host end includes: A second determination module, configured to determine each first sample from each sample in the Host end, and determine a first feature vector corresponding to each first sample and a weight corresponding to the first feature vector; The second determination module is used to determine the first parameter according to the first feature vector and the weight corresponding to the first feature vector, and send the first parameter and the identifier of the first sample corresponding to the first parameter to the guest end; wherein the guest end is a bank terminal having samples and labels corresponding to the samples, and the host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is represented by a feature vector; A second receiving module, configured to receive the intermediate parameters fed back by the guest, and determine a second gradient according to the intermediate parameters and each of the first feature vectors, so as to complete the vertical federated learning; A second sending module is used to generate a second sending information according to the second gradient and the second number of the first samples, and send the second sending information to a third-party terminal, wherein the third-party terminal determines a second parameter model according to the first sending information sent by each of the Guest terminals and the second sending information sent by each of the Host terminals to complete horizontal federated learning; wherein the Guest terminal determines a first combined feature vector based on each of the first gradients, encrypts the first combined feature vector using a first encryption algorithm, generates the first sending information based on the encrypted first combined feature vector and the first number of the second samples, and sends the first sending information to the third-party terminal, the Host terminal determines a second combined feature vector based on each of the second gradients, encrypts the second combined feature vector using a second encryption algorithm, and generates the second sending information based on the encrypted second combined feature vector and the second number of the first samples, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; The second receiving module is further used to receive the second model parameters fed back by the third-party terminal, and update the parameters of the second model according to the second model parameters, and train the second model after the updated parameters according to each of the first samples to obtain a second prediction model; wherein, after receiving the second model parameters, the second model parameters are decrypted, and the parameters of the second model are updated according to the decrypted second model parameters; After obtaining the second prediction model, the Host performs risk prediction on the user, wherein a sample of the second user to be predicted is obtained, and a fourth eigenvector is extracted from the sample of the second user to be predicted, the fourth eigenvector is input into the second prediction model, a second numerical value is output, and a prediction result of the second user is determined based on the second numerical value.

16. A third-party terminal, characterized in that: The third-party terminal includes: A second acquisition module is used to acquire the first transmission information sent by each Guest end and the second transmission information sent by each Host end, and acquire the first gradient and the first number of the second samples according to the first transmission information, and acquire the second gradient and the second number of the first samples according to the second transmission information; wherein the Guest end is a terminal of a bank having samples and labels corresponding to the samples, and the Host end is a terminal of an Internet company having only samples but not labels of the samples, wherein the samples correspond to multiple features, and each feature is represented by a feature vector; wherein the Guest end determines a first combined feature vector based on each first gradient, and encrypts the first combined feature vector using a first encryption algorithm, generates a first transmission information based on the encrypted first combined feature vector and the first number of the second samples, and sends the first transmission information to a third-party terminal, the Host end determines a second combined feature vector based on each second gradient, and encrypts the second combined feature vector using a second encryption algorithm, generates a second transmission information based on the encrypted second combined feature vector and the second number of the first samples, and sends the second mode information to the third-party terminal, wherein the first encryption algorithm and the second encryption algorithm are homomorphic encryption algorithms; A third determination module, configured to determine a model parameter to be processed according to the first gradient, the first quantity, the second gradient, and the second quantity; A generating module, configured to generate first model parameters corresponding to the Guest end according to the model parameters to be processed and the first quantity corresponding to the Guest end, and to generate second model parameters corresponding to the Host end according to the model parameters to be processed and the second quantity corresponding to the Host end; The third sending module is used to send the first model parameter to the Guest end corresponding to the first model parameter, and send the second model parameter to the Host end corresponding to the second model parameter.

17. An electronic device, characterized in that: The electronic device includes a memory, a processor, and instructions or programs stored in the memory and executable on the processor. When the instructions or programs are executed by the processor, the steps of the model training method of vertical federated learning and horizontal federated learning are implemented as described in any one of claims 1-13.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a program or an instruction, and when the program or the instruction is executed by a processor, the steps of the model training method of vertical federated learning and horizontal federated learning as described in any one of claims 1-13 are implemented.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the model training method of vertical federated learning and horizontal federated learning as described in any one of claims 1 to 13 are implemented.

Images