A longitudinal federated model training method and system

Abstract

The application discloses a longitudinal federal model training method and system, which is applied to a first data party with labels, and the method comprises the following steps: calculating sample feature gradients of all samples based on labels; receiving a feature ranking table sent by a second data party; integrating the sample feature gradients and the feature ranking table to obtain gradient statistical information; and updating a model of the first data party according to the gradient statistical information and feature identification. The application can realize the training of the model of the first data party only through the sample feature gradients of the first data party and the feature ranking table of the second data party, avoids the repeated encryption and decryption operation in the existing federal algorithm, simplifies the model training process, reduces the communication frequency, improves the training efficiency, reduces the risk of data leakage in the data exchange process due to the absence of multiple data exchanges, and improves the security.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, in particular to a longitudinal federated model training method and system. BACKGROUND

[0002] The rapid development of technologies such as the Internet, big data, and artificial intelligence has brought convenience and efficiency to people's lives. At the same time, it has also brought about the generation of massive amounts of data and the expansion of data assets. The development and utilization of data, value mining, and cross-border flow of data production activities have brought great challenges to data security and personal information protection, and may be accompanied by risks such as endangering national security, public interest, or personal privacy rights and interests.

[0003] Federated learning (FL) is a machine learning framework that enables machine learning modeling by combining data from multiple participants while meeting data privacy and security requirements and government regulations. It effectively solves the "data silo" problem.

[0004] Federated learning can use multi-party data to achieve joint modeling, which helps to break the "data silo" situation and greatly improves the ability of data circulation and machine learning models. However, in the existing federated learning process, there are still many communication time consumptions generated by multi-party data circulation, which reduces the running algorithm efficiency, increases the risk of data privacy leakage, and lowers the accuracy of model prediction results. SUMMARY

[0005] Therefore, the embodiments of the present application provide a longitudinal federated model training method to solve the problems of low precision, low efficiency, and security risks in the prior art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] The embodiments of the present application provide a longitudinal federated model training method applied to a first data party with labels, comprising:

[0008] Calculating sample feature gradients of all samples based on labels;

[0009] Receiving a feature ranking table sent by a second data party, wherein the feature ranking table is obtained by analyzing feature data of the second data party to obtain feature identifiers, and the feature identifiers are sorted according to the sizes of feature values of the feature data;

[0010] Integrating the sample feature gradients and the feature ranking table to obtain gradient statistical information;

[0011] Updating a model of the first data party according to the gradient statistical information and the feature identifiers.

[0012] Optionally, the integrating the sample feature gradient and the feature ranking table to obtain gradient statistical information comprises:

[0013] Obtaining a feature identifier of each sample feature gradient;

[0014] Comparing the feature identifier with a feature identifier column in the feature ranking table to obtain an arrangement order of the sample feature gradient;

[0015] Ranking the sample feature gradient according to the arrangement order to obtain gradient statistical information.

[0016] Optionally, the updating the model of the first data party according to the gradient statistical information and the feature identifier comprises:

[0017] Finding an optimal split point from the gradient statistical information according to the feature identifier to split a node until an end node;

[0018] Calculating weight information of the end node according to the sample feature gradient;

[0019] Updating sample feature gradients of all samples based on the weight information, and returning to the step of integrating the sample feature gradient and the feature ranking table to obtain gradient statistical information until a preset tree building condition is not met to complete the updating of the model of the first data party.

[0020] Optionally, the process of finding the optimal split point to split the node comprises:

[0021] Obtaining a feature point that can be used as a split point according to the feature identifier;

[0022] Selecting an optimal split point from the feature point;

[0023] Judging a source of the optimal split point;

[0024] If the optimal split point belongs to the first data party, splitting the node according to a feature value of the optimal split point;

[0025] If the optimal split point belongs to the second data party, splitting the node according to a feature identifier of the optimal split point.

[0026] Optionally, the selecting the optimal split point from the feature point comprises:

[0027] Calculating optimal split point information in each feature point by a preset algorithm;

[0028] If the optimal split point information belongs to the second data party, the optimal split point information is {id, score_ij}, wherein id is a feature identifier, and score_ij represents a loss difference of the jth feature of the ith participant;

[0029] If the optimal split point information belongs to the first data party, the optimal split point information is {split_value, score_ij}, wherein split_value is a feature value of the optimal split point, and score_ij represents a loss difference of the jth feature of the ith participant.

[0030] By comparing the loss differences in the optimal split point information of all feature points, the feature point with the maximum loss difference can be obtained as the optimal split point.

[0031] Optionally, the method further comprises:

[0032] Filtering node information of the updated model;

[0033] Hiding split information in the node information that does not belong to the second data party;

[0034] Sending the model after the hiding processing to the second data party.

[0035] The embodiment of the application provides a longitudinal federated model training method, applied to a second data party without labels, comprising:

[0036] Analyzing feature data of itself to obtain feature identifiers;

[0037] Sorting all feature identifiers according to sizes of feature values of the feature data to obtain a feature sorting table;

[0038] Sending the feature sorting table to a first data party with labels, so that the first data party updates a model of the first data party by using the feature sorting table.

[0039] Optionally, the method further comprises:

[0040] Receiving a model after hiding processing sent by the first data party, wherein the model after hiding processing is obtained by filtering node information of the updated model by the first data party, and hiding split information in the node information that does not belong to the second data party; and the gradient statistical information is obtained by integrating sample feature gradients of all samples calculated based on labels by the first data party and the feature sorting table;

[0041] Extracting node information of each feature point from the model after hiding processing;

[0042] updating a model of the second data party based on the node information and the feature ranking table.

[0043] Optionally, the updating the model of the second data party based on the node information and the feature ranking table comprises:

[0044] judging the source of the feature point based on a split mode in the node information;

[0045] if the feature point belongs to the second data party, updating the model of the second data party according to the feature identification and the feature ranking table.

[0046] The embodiment of the application further provides a vertical federated model training system, comprising a first data party and a second data party,

[0047] The first data party calculates sample feature gradients of all samples based on labels.

[0048] The second data party analyzes feature data of itself to obtain feature identifications, ranks all the feature identifications according to the sizes of feature values of the feature data to obtain a feature ranking table, and sends the feature ranking table to the first data party.

[0049] The first data party receives the feature ranking table sent by the second data party, integrates the sample feature gradients and the feature ranking table to obtain gradient statistical information, and updates a model of the first data party according to the gradient statistical information and the feature identifications.

[0050] The embodiment of the application further provides an electronic device, comprising:

[0051] A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the vertical federated model training method provided by the embodiment of the application.

[0052] The embodiment of the application further provides a computer readable storage medium, which stores computer instructions for making a computer execute the vertical federated model training method provided by the embodiment of the application.

[0053] The technical scheme of the application has the following advantages:

[0054] The application provides a longitudinal federal model training method, which is applied to a first data party with labels, and sample feature gradients of all samples are calculated based on the labels; a feature sorting table is received, which is sent by a second data party, the feature sorting table is obtained by analyzing feature data of the second data party, and feature identifiers are obtained; all the feature identifiers are sorted according to the sizes of feature values of the feature data to obtain the feature sorting table; the feature sequence is marked based on the feature identifiers; and gradient statistical information is obtained by integrating the sample feature gradients and the feature sorting table; and the model of the first data party is updated according to the gradient statistical information and the feature identifiers. The model of the first data party can be trained only by the sample feature gradients of the first data party and the feature sorting table of the second data party, the operation of repeated encryption and decryption in the existing federal algorithm is avoided, the model training process is simplified, the communication frequency is reduced, the training efficiency is improved, the risk of data leakage in the data exchange process is reduced, and the security is improved.

[0055] The application provides a longitudinal federal model training method, which is applied to a second data party without labels, feature identifiers are obtained by analyzing feature data of the second data party; all the feature identifiers are sorted according to the sizes of feature values of the feature data to obtain a feature sorting table; and the feature sorting table is sent to a first data party with labels, so that the first data party updates the model of the first data party by using the feature sorting table. The model of the first data party can be updated only by generating a feature sequence and sending the feature sequence to the first data party, only one communication is needed, the training process is greatly simplified, the feature sequence sent is a feature identifier shared by the longitudinal federal multiple parties, the feature identifier does not leak any information, and the risk of feature information leakage can be reduced.

[0056] The application provides a longitudinal federal model training system, which comprises a first data party and a second data party, the first data party calculates sample feature gradients of all samples based on labels; the second data party analyzes feature data of the second data party to obtain feature identifiers; all the feature identifiers are sorted according to the sizes of feature values of the feature data to obtain a feature sorting table; the feature sorting table is sent to the first data party; the first data party receives the feature sorting table sent by the second data party; gradient statistical information is obtained by integrating the sample feature gradients and the feature sorting table; and the model of the first data party is updated according to the gradient statistical information and the feature identifiers. The model of the first data party can be trained only by the sample feature gradients of the first data party and the feature sorting table of the second data party, the operation of repeated encryption and decryption in the existing federal algorithm is avoided, the model training process is simplified, the communication frequency is reduced, the training efficiency is improved, the risk of data leakage is reduced, and the security is improved. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the specific embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the specific embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0058] Figure 1 The flowchart of the longitudinal federal model training method in the embodiments of the present application;

[0059] Figure 2 The flowchart of integrating the sample feature gradient and the feature ranking table according to the embodiments of the present application;

[0060] Figure 3 The flowchart of updating the first data party model according to the embodiments of the present application;

[0061] Figure 4 The flowchart of finding the optimal split point to split the node according to the embodiments of the present application;

[0062] Figure 5 The flowchart of selecting the optimal split point according to the embodiments of the present application;

[0063] Figure 6 The flowchart of sending the model after hidden processing to the second data party according to the embodiments of the present application;

[0064] Figure 7 The flowchart of the longitudinal federal model training method in the embodiments of the present application;

[0065] Figure 8 The flowchart of updating the model of the second data party based on the model after hidden processing according to the embodiments of the present application;

[0066] Figure 9 The flowchart of updating the model of the second data party according to the embodiments of the present application;

[0067] Figure 10 The structural schematic diagram of the electronic device in the embodiments of the present application. DETAILED DESCRIPTION

[0068] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0069] According to the embodiments of the present application, a longitudinal federal model training method embodiment is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0070] A longitudinal federal model training method is provided in the present embodiment, which can be applied to a labeled first data party. According to the longitudinal federal definition, multiple participants have cross samples, but the feature information owned by the participants is different. The first data party and the second data party have the same sample, and therefore have the same feature identifier (sample ID). As shown in the figure, the longitudinal federal model training method includes the following steps: Figure 1

[0071] Step S11: calculating the sample feature gradient of all samples based on the label. Specifically, the sample feature gradient includes the first-order gradient and the second-order gradient. The first-order gradient and the second-order gradient belong to sensitive information, and there is a risk of data leakage, so they cannot be directly transmitted.

[0072] Step S12: receiving the feature sorting table sent by the second data party. The feature sorting table is obtained by the second data party analyzing its own feature data to obtain feature identifiers, and then sorting all the feature identifiers according to the size of the feature values of the feature data. Specifically, the second data party only needs to generate a feature sequence and send it to the first data party, so that the first data party can complete the model update through only one communication, greatly simplifying the training process.

[0073] Step S13: integrating the sample feature gradient and the feature sorting table to obtain gradient statistical information. Specifically, the first data party can obtain a sample feature gradient table that maintains the sorting information by integrating the feature sorting table, thereby obtaining the gradient statistical information of all features of both parties.

[0074] Step S14: updating the model of the first data party according to the gradient statistical information and the feature identifier.

[0075] ​Through the steps S11 to S14, the longitudinal federation model training method provided by the embodiment of the present application can realize the training of the self model only through the sample feature gradient of the self and the feature ranking table of the second data party, avoid the repeated encryption and decryption operation in the existing federation algorithm, simplify the model training process, reduce the communication frequency, improve the training efficiency, reduce the data leakage risk in the data exchange process due to the absence of multiple data exchanges, and improve the security.

[0076] Specifically, in an embodiment, the step S13 is specifically as shown in the following step S131 to step S134. Figure 2

[0077] Step S131: Obtain the feature identifier of each sample feature gradient.

[0078] Step S132: Extract the feature identifier column and the feature data column from the feature ranking table.

[0079] Step S133: Compare the feature identifier with the feature identifier in the feature ranking table to obtain the arrangement order of the sample feature gradient.

[0080] Step S134: Sort the sample feature gradient according to the arrangement order to obtain the gradient statistical information.

[0081] Specifically, after the first data party receives the feature pre-ranking list sent by the second data party, according to the concept of longitudinal federation, the first data party and the second data party have the same feature identifier and different features, and the first data party can correspond the feature identifier of each feature in the feature pre-ranking list sent by the second data party with the feature identifier of the sample feature gradient of each sample. For each feature of the second data party, the first data party can obtain an arrangement order table maintaining the sorting information through integration. Thus, the first data party obtains the gradient statistical information of all features. This process can be completed only through simple comparison and sorting, and the encryption and decryption and ciphertext operation of the prior art need to consume a large amount of time, especially when the data volume is large. The encryption and decryption and ciphertext operation operation will damage the data precision and the model prediction precision. Therefore, this process can effectively improve the model precision, reduce the calculation amount, save time and improve the efficiency.

[0082] Specifically, in an embodiment, the step S14 is specifically as shown in the following step S141 to step S142. Figure 3

[0083] Step S141: Find the optimal split point from the gradient statistical information according to the feature identifier to perform node splitting until the end node.

[0084] Step S142: Calculate the weight information of the end node according to the sample feature gradient. ​​

[0085] Step S143: updating the sample feature gradient of all samples based on the weight information, and returning to the step of integrating the sample feature gradient and the feature ranking table to obtain the gradient statistical information until the preset tree building condition is not met to complete the updating of the model of the first data party.

[0086] Specifically, the optimal weight of the end node is only related to the first-order gradient and the second-order gradient, which is calculated by the first data party. The node does not synchronize, and the second data party cannot know the weight of the end node. The first data party calculates the first-order gradient g and the second-order gradient h of each instance according to the weight information of the end node of the current tree. The above process is repeated in the next tree until the preset tree building condition is not met. Only the first data party performs the operation of this process, which greatly reduces the data calculation amount compared with the round-trip operation of multiple encryption and decryption, and improves the overall efficiency.

[0087] Specifically, in an embodiment, the process of finding the optimal split point for node splitting in step S141 described above, as shown in FIG. 6, specifically includes the following steps: Figure 4

[0088] Step S1411: obtaining the feature points that can be used as split points according to the feature identifiers.

[0089] Step S1412: selecting the optimal split point from the feature points.

[0090] Step S1413: judging the source of the optimal split point.

[0091] Step S1414: if the optimal split point belongs to the first data party, splitting the node according to the feature value of the optimal split point.

[0092] Step S1415: if the optimal split point belongs to the second data party, splitting the node according to the feature identifier of the optimal split point.

[0093] Specifically, for the features of the first data party, the selection of the candidate split point and the finding of the optimal split point can adopt Exact Greedy Algorithm or Approximate Algorithm. For the features of the second data party, there are three ways to select the split point:

[0094] ​Method 1: Splitting for each feature identifier of all features of the second data party once, and calculating the optimal loss error. Specifically, since the first data party has no information of the feature pre-sorting gradient statistics table of the second data party. Therefore, the first data party cannot obtain the candidate split point through the Exact Greedy Algorithm method, and thus can adopt the method of splitting all feature identifiers. This process has the highest accuracy.

[0095] Method 2: The second data party adds a candidate split point marking column when constructing the feature sorting table, that is, the second data party needs to send a feature and sorting list [Xi1_sorted_id, Xi1_split_tag, Xi2_sorted_id, Xi2_split_tag…, Xiq_sorted_id, Xiq_split_tag] list with split point information to the first data party. It can be seen that for each feature identifier column of the feature sorting, a split point marking column Xij_sorted_id is accompanied, which is a sparse list. The feature sorting list [feature_value, id, split_tag], wherein the first column feature_value is the value sorting of the current feature, the second column is the feature identifier of the corresponding cross sample, and the third column is the marker of whether it is a split point. The second data party removes the first column feature value information of the list to obtain an instance [id, split_tag] list sorted according to the feature value. After the second data party completes the feature traversal, a feature pre-sorting list with candidate split points [Xi1_sorted_id, Xi1_split_tag, Xi2_sorted_id, Xi2_split_tag…, Xiq_sorted_id, Xiq_split_tag] is obtained. Based on this, the first data party can adopt the Exact Greedy Algorithm to obtain the candidate split point. This process can reduce part of the calculation amount under the condition of ensuring a certain accuracy.

[0096] Method 3: After the first data obtains the feature sorting table of the second data party, it can directly adopt the Approximate Algorithm, without adding the candidate split point marking list. Since the selection of the candidate split point in the Approximate Algorithm is obtained through the weight quantile point technology, it requires taking the second order gradient h as the weight of each feature identifier. Similarly, the second order gradient weight h only exists in the first data party, so the first data has all the conditions to implement the weight quantile point technology, and thus can directly obtain the optimal split point through the Approximate Algorithm. This process has low calculation amount and can improve the overall efficiency.

[0097] After finding the optimal split point, if the feature belongs to the first data cube, the first data cube performs a node split based on the feature value split_value, saves the node partition information {Xij, split_value}, and partitions the pre-sorted gradient statistics of the second data cube's features based on the feature identifier. Node synchronization is not required. If the feature belongs to the second data cube, the first data cube partitions the feature gradient statistics table of the second data cube based on the feature identifier and saves the node partition information {Xij, id}. Node synchronization is not required.

[0098] You can select a suitable process from the above methods to obtain the optimal splitting point according to your needs, providing users with more choices.

[0099] Specifically, in one embodiment, the above step S1412 is as follows: Figure 5 As shown, the specific steps include:

[0100] Step S14121: Calculate the optimal splitting point information for each feature point using a preset algorithm. Specifically, for the features of the first data cube, the selection of candidate splitting points and the search for the optimal splitting point can use the Exact Greedy Algorithm or the Approximate Algorithm. For the features of the second data cube, the above three methods can be used for calculation.

[0101] Step S14122: If the optimal splitting point information belongs to the second data party, the optimal splitting point information is {id, score_ij}, where id is the feature identifier and score_ij represents the loss difference of the j-th feature of the i-th participant.

[0102] Step S14123: If the optimal splitting point information belongs to the first data party, the optimal splitting point information is {split_value, score_ij}, where split_value is the feature value for obtaining the optimal splitting point, and score_ij represents the loss difference of the jth feature of the i-th participant.

[0103] Step S14124: Compare the loss differences in the optimal splitting point information of all feature points, and obtain the feature point with the largest loss difference as the optimal splitting point.

[0104] Specifically, through this process, the optimal splitting point can be quickly and effectively selected, and the effective splitting point information can be stored.

[0105] Specifically, in one embodiment, the above-mentioned vertical federated model training method, such as Figure 6 As shown, the following steps are also included:

[0106] Step S15: Filter the node information of the updated model.

[0107] Step S16: Hide the split information in the node information that does not belong to the second data party.

[0108] Step S17: Send the hidden model to the second data party.

[0109] Specifically, by generating a second data party with the hidden model, the second data party can update the model. This process is the second communication. Compared with the existing technology, it greatly reduces the number of communications, reduces the time consumption caused by encryption and decryption and ciphertext operations, and improves the efficiency of model training.

[0110] In this embodiment, a longitudinal federated model training method is also provided, which can be applied to unlabeled second data, such as Figure 7 As shown, the vertical federation model training method includes the following steps:

[0111] Step S21: Analyze the feature data of the device to obtain a feature identifier.

[0112] Step S22: sorting all feature identifiers according to the size of the feature values ​​of the feature data to obtain a feature sorting table.

[0113] Step S23: Send the feature sorting table to the first data party with labels, so that the first data party uses the feature sorting table to update the model of the first data party. Specifically, the second data party has no labels and cannot independently construct the gradient statistical information table of the features. The second data party performs the following operations on each of its own features: sort the individual features according to the size of the feature values, and obtain a feature sorting list [feature_value, id], where the first column feature_value is the value sorting of the current feature, and the second column is the feature identifier (corresponding cross-user instance id). The second data party removes the first column of feature value information from the list to obtain a feature identifier list sorted according to the feature value. After the feature traversal of the second data party is completed, a feature sorting table [Xi1_sorted_id, Xi2_sorted_id,…, Xiq_sorted_id] is obtained, where Xi_sorted_id identifies the feature identifier column after the i-th feature is sorted according to its feature value. This feature pre-sorting list is sent to the first data party.

[0114] Through the steps S21 to S23, the longitudinal federal model training method provided by the embodiment of the present application only needs to generate a feature sequence and send the feature sequence to the first data party, so that the first data party can complete model updating, and only one communication is needed, which greatly simplifies the training process, and the feature sequence sent is a feature identifier shared by the longitudinal federal multiple parties, and the feature identifier does not leak any information, so that the risk of feature information leakage can be reduced.

[0115] Specifically, in an embodiment, the longitudinal federal model training method described above, as shown in Figure 8 , further includes the following steps:

[0116] Step S24: receiving the hidden processed model sent by the first data party, the hidden processed model being obtained by the first data party filtering node information of the updated model, and the split information in the node information not belonging to the second data party being hidden; the gradient statistical information being obtained by the first data party calculating sample feature gradients of all samples based on labels and integrating the sample feature gradients and the feature ranking table.

[0117] Step S25: extracting node information of each feature point from the hidden processed model.

[0118] Step S26: updating the model of the second data party based on the node information and the feature ranking table.

[0119] Specifically, after the second data party receives the hidden processed model, each tree node information of the model is traversed one by one. For the node whose feature belongs to the party, the corresponding optimal split feature value can be found according to the feature ranking table [feature_value, id] by comparing {Xij, id}, and the node division information {Xij, feature_value} is obtained. If the current feature does not belong to the party, no processing is performed. After all nodes of the model are traversed, the second data party updates the current model. The model of the party can be updated only by receiving the hidden processed model, which greatly improves the time, communication times, model accuracy and efficiency compared with the prior art.

[0120] Specifically, in an embodiment, the step S26 described above, as shown in Figure 9 , specifically includes the following steps:

[0121] Step S261: judging the source of the feature point based on the split mode in the node information.

[0122] Step S262: if the feature point belongs to the second data party, updating the model of the second data party according to the corresponding feature value obtained from the feature identifier and the feature ranking table.

[0123] Specifically, the data processing update is performed by traversing the own characteristic identifier, the process is simple, and the update can be quickly and efficiently completed.

[0124] In the embodiment, a longitudinal federal model training system is also provided, comprising a first data party and a second data party,

[0125] The first data party calculates sample feature gradients of all samples based on labels;

[0126] The second data party analyzes own feature data to obtain characteristic identifiers, sorts all characteristic identifiers according to sizes of characteristic values of the feature data to obtain a feature sorting table, and sends the feature sorting table to the first data party;

[0127] The first data party receives the feature sorting table sent by the second data party, integrates the sample feature gradients and the feature sorting table to obtain gradient statistical information, and updates a model of the first data party according to the gradient statistical information and the characteristic identifiers.

[0128] Specifically, the model of the first data party can be trained according to the sample feature gradients and the feature sorting table of the second data party through only one data transmission, the operation of repeated encryption and decryption in the existing federal algorithm is avoided, the model training process is simplified, the communication frequency is reduced, the training efficiency is improved, the risk of data leakage in the data exchange process is reduced due to the absence of multiple data exchanges, and the security is improved.

[0129] According to the embodiments of the present application, an electronic device is also provided, as shown in the figure. Figure 10 The electronic device can include a processor 901 and a memory 902, wherein the processor 901 and the memory 902 can be connected by a bus or other means, Figure 10 For example, the bus connection.

[0130] The processor 901 can be a central processing unit (CPU). The processor 901 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or combinations of the above various types of chips.

[0131] The memory 902, as a non-transitory computer readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the methods in the method embodiments of the present application. The processor 901 performs various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 902, that is, implements the methods in the above method embodiments.

[0132] The memory 902 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function; and the data storage area can store data created by the processor 901 and the like. In addition, the memory 902 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 902 can optionally include a memory disposed remotely with respect to the processor 901, which can be connected to the processor 901 through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0133] One or more modules are stored in the memory 902, and when executed by the processor 901, perform the methods in the above method embodiments.

[0134] The above electronic device specific details can be understood in correspondence with the above method embodiments corresponding to the relevant description and effects, which will not be described here.

[0135] Those skilled in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The program can be stored in a computer readable storage medium, and when executed, can include the processes of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage medium can also include a combination of the above types of memories.

[0136] Although the embodiments of the present application are described in conjunction with the accompanying drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A method for training a vertical federated model, applied to a labeled first data cube, characterized in that: include: Calculate the sample feature gradients of all samples based on the labels; Receive a feature ranking table sent by the second data party, wherein the feature ranking table is obtained by analyzing the feature data of the second data party to obtain feature identifiers; and sorting all feature identifiers according to the size of the feature values ​​of the feature data; Integrating the sample feature gradient and the feature ranking table to obtain gradient statistical information; Updating the model of the first data cube according to the gradient statistical information and the feature identifier; The step of integrating the sample feature gradient and the feature ranking table to obtain gradient statistical information includes: Obtain the feature identity of each sample feature gradient; Comparing the feature identifier with the feature identifier in the feature ranking table to obtain an arrangement order of the sample feature gradients; The sample feature gradients are sorted according to the arrangement order to obtain gradient statistical information.

2. The vertical federated model training method according to claim 1, characterized in that: The updating of the model of the first data cube according to the gradient statistical information and the feature identifier includes: Searching for an optimal splitting point from the gradient statistical information according to the feature identifier to perform node splitting until reaching the terminal node; Calculating weight information of the terminal node according to the sample feature gradient; Based on the weight information, the sample feature gradients of all samples are updated, and the step of integrating the sample feature gradients and the feature ranking table to obtain gradient statistical information is returned until the preset tree building condition is not met and the model of the first data cube is updated.

3. The vertical federated model training method according to claim 2, characterized in that: The process of finding the optimal splitting point to split the node includes: Obtaining feature points that can be used as splitting points according to the feature identifier; Selecting an optimal splitting point from the feature points; Determining the source of the optimal splitting point; If the optimal splitting point belongs to the first data cube, performing node splitting on the optimal splitting point according to the eigenvalue of the optimal splitting point; If the optimal splitting point belongs to the second data cube, the optimal splitting point is node split according to the feature identifier of the optimal splitting point.

4. The vertical federated model training method according to claim 3, characterized in that: The selecting the optimal splitting point from the feature points includes: The optimal splitting point information of each feature point is calculated by a preset algorithm; If the optimal splitting point information belongs to the second data entity, then the optimal splitting point information is {id, score_ij}, where id is the feature identifier and score_ij is the loss difference of the jth feature of the i-th participant; If the optimal splitting point information belongs to the first data party, then the optimal splitting point information is {split_value, score_ij}, where split_value is the feature value for obtaining the optimal splitting point, and score_ij is the loss difference of the jth feature of the i-th participant; By comparing the loss differences in the optimal splitting point information of all feature points, the feature point with the largest loss difference can be obtained as the optimal splitting point.

5. The vertical federated model training method according to claim 1, characterized in that: The method further comprises: Filter the node information of the updated model; Hiding the split information in the node information that does not belong to the second data party; The hidden processed model is sent to the second data cube.

6. A method for training a longitudinal federated model, applied to an unlabeled second data cube, characterized in that: include: Analyze its own feature data to obtain feature identification; Sort all feature identifiers according to the size of the feature values ​​of the feature data to obtain a feature sorting table; Sending the feature ranking table to the first data cube with the label, so that the first data cube updates the model of the first data cube by using the feature ranking table; The method further comprises: Receive the hidden model sent by the first data party, where the hidden model is obtained by the first data party filtering the node information of the updated model and hiding the split information in the node information that does not belong to the second data party; the gradient statistical information is obtained by the first data party calculating the sample feature gradients of all samples based on the labels and integrating the sample feature gradients with the feature ranking table; Extracting node information of each feature point from the hidden processed model; The model of the second data cube is updated based on the node information and the feature ranking table.

7. The vertical federated model training method according to claim 6, characterized in that: The updating of the model of the second data cube based on the node information and the feature ranking table includes: Determining the source of the feature point based on the splitting method in the node information; If the feature point belongs to the second data cube, the model of the second data cube is updated by obtaining the corresponding feature value according to the feature identifier and the feature sorting table.

8. A vertical federated model training system, characterized by: include: The first data party and the second data party, The first data entity calculates the sample feature gradients of all samples based on the labels; The second data party analyzes its own feature data to obtain feature identifiers; sorts all feature identifiers according to the size of the feature values ​​of the feature data to obtain a feature sorting table; and sends the feature sorting table to the first data party; The first data party receives the feature ranking table sent by the second data party; integrates the sample feature gradient and the feature ranking table to obtain gradient statistical information; and updates the model of the first data party according to the gradient statistical information and the feature identifier; The step of integrating the sample feature gradient and the feature ranking table to obtain gradient statistical information includes: Obtain the feature identity of each sample feature gradient; Comparing the feature identifier with the feature identifier in the feature ranking table to obtain an arrangement order of the sample feature gradients; The sample feature gradients are sorted according to the arrangement order to obtain gradient statistical information.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the vertical federated model training method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the longitudinal federated model training method according to any one of claims 1 to 7.

Images