A model training method, device, medium and electronic equipment

Abstract

The application provides a model training method and device, a medium and an electronic device, including: obtaining initiator task training samples; repeating the following steps: inputting the initiator task training samples into an initiator bottom-layer model to obtain first intermediate feature data; receiving second intermediate feature data sent by each collaboration terminal; obtaining fusion feature data according to the first intermediate feature data and the second intermediate feature data; inputting the fusion feature data into an upper-layer model of a to-be-trained vertical federated learning model to determine a target task loss value; respectively determining a first training bias of the first intermediate feature data and a second training bias of each second intermediate feature data; updating model parameters of a task terminal bottom-layer model and the upper-layer model based on the target task loss value and the first training bias, and instructing each collaboration terminal to update model parameters of a corresponding collaboration terminal bottom-layer model based on the second training bias, until the target task loss value converges to a preset target value.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data security technology, and in particular to a model training method, device, medium and electronic device. Background Art

[0002] Recently, federated learning has become a research focus in the field of privacy-preserving computing. Federated learning allows data to be distributed across different devices and networks, making them invisible to each other. This addresses security issues such as privacy leaks that exist in data collaboration tasks. Vertical federated learning models heterogeneous data and is primarily used for data collaboration between organizations such as businesses, governments, and institutions. It has a wide range of applications. In vertical federated learning, each participant shares common sample objects, but the feature categories of the samples differ. Only one participant, known as the task initiator, provides task label information, while the other participants are known as data providers. Through vertical federated learning, the task initiator can utilize more comprehensive feature information from the samples while ensuring data security, improving the accuracy of model predictions.

[0003] In related technologies, vertical federated learning is usually single-task, involving only one task initiator (Guest) and multiple data providers (Host). The task roles of the task initiator and the data provider are fixed, and the task initiator only uses labeled samples for training. In actual scenarios, if the data provider also has business modeling needs and requires the sample features of the task initiator, it is necessary to re-initiate the federated modeling task, which is time-consuming and costly. Therefore, the model training process in multi-task vertical federated learning is less efficient. Summary of the Invention

[0004] The present application provides a model training method, apparatus and equipment that can realize multi-task parallel vertical federated learning, and can initiate training on multiple task terminals in parallel, thereby improving the model training efficiency of the multi-task vertical federated learning process.

[0005] In a first aspect, an embodiment of the present application provides a model training method, which is applied to a target task initiating terminal of a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the method includes:

[0006] Obtain initiator task training samples;

[0007] Repeat the following steps:

[0008] Inputting the initiator task training sample into the initiator underlying model of the longitudinal federated learning model to be trained, and obtaining first intermediate feature data output by the initiator underlying model;

[0009] Receiving second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the cooperating terminal is selected from the multiple participating terminals; the second intermediate feature data is intermediate feature data obtained by the cooperating terminal underlying model of the longitudinal federated learning model to be trained of each cooperating terminal based on the cooperating terminal task training samples stored locally;

[0010] Obtaining fused feature data of a training sample of a target task based on the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiating terminal;

[0011] Inputting the fused feature data into an upper model of the longitudinal federated learning model to be trained to determine a target task loss value;

[0012] respectively determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data;

[0013] Based on the target task loss value and the first training deviation, the model parameters of the task-side underlying model and the upper-layer model are updated, and based on the second training deviation, each of the collaborative terminals is instructed to update the model parameters of the corresponding collaborative-side underlying model until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

[0014] A second aspect of the present application provides a model training method, which is applied to a target collaborative terminal corresponding to a target task initiating terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the target collaborative terminal is any one of the collaborative terminals of the target task initiating terminal; the collaborative terminal is selected from the multiple participating terminals; the method includes:

[0015] Obtaining a collaborative terminal task training sample of the target collaborative terminal;

[0016] Repeat the following steps:

[0017] Inputting the collaborative end task training sample into the collaborative end underlying model of the longitudinal federated learning model to be trained, and obtaining second intermediate feature data output by the collaborative end underlying model;

[0018] The second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the target task is the task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by the target task initiating terminal inputting the initiating-end task training sample into the task-end underlying model of the longitudinal federated learning model to be trained;

[0019] In response to the instruction of the target task initiating terminal, the model parameters of the underlying model of the collaborative end are updated until the target task loss value converges to a preset target value, thereby obtaining a trained vertical federated learning model; the instruction is issued by the target task initiating terminal when determining the target task loss value based on the fused feature data, determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each of the collaborative terminals, and updating the model parameters of the task end underlying model and the upper-layer model.

[0020] A third aspect of the present application provides a model training device, which is applied to a target task initiating terminal of a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the device includes:

[0021] The task-side reading unit is used to obtain the initiator-side task training samples;

[0022] The task-side training unit is used to repeatedly perform the following steps:

[0023] Inputting the initiator task training sample into the initiator underlying model of the longitudinal federated learning model to be trained, and obtaining first intermediate feature data output by the initiator underlying model;

[0024] Receiving second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the cooperating terminal is selected from the multiple participating terminals; the second intermediate feature data is intermediate feature data obtained by the cooperating terminal underlying model of the longitudinal federated learning model to be trained of each cooperating terminal based on the cooperating terminal task training samples stored locally;

[0025] Obtaining fused feature data of a training sample of a target task based on the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiating terminal;

[0026] Inputting the fused feature data into an upper model of the longitudinal federated learning model to be trained to determine a target task loss value;

[0027] respectively determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data;

[0028] Based on the target task loss value and the first training deviation, the model parameters of the task-side underlying model and the upper-layer model are updated, and based on the second training deviation, each of the collaborative terminals is instructed to update the model parameters of the corresponding collaborative-side underlying model until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

[0029] In an optional embodiment, the task-side training unit is specifically configured to:

[0030] respectively determining the sample object identifiers corresponding to the first intermediate feature data and each second intermediate feature data;

[0031] The first intermediate feature data and each second intermediate feature data corresponding to the same sample object identifier are respectively spliced ​​in a preset order to obtain fused feature data of the training sample of the target task.

[0032] In an optional embodiment, the device further includes a sample setting unit, configured to:

[0033] Selecting a first preset number of participating terminals from participating terminals other than the target task initiating terminal as cooperative terminals of the target task initiating terminal;

[0034] The intersection samples are determined by private intersection with each of the collaborative terminals to obtain a training sample set for the target task; the training sample set includes the initiating end task training samples stored in the target task initiating terminal, and the collaborative end task training samples respectively stored in each of the collaborative terminals of the target task initiating terminal.

[0035] In an optional embodiment, the task-side training unit is specifically configured to:

[0036] Adjust the model parameters of the upper model of the longitudinal federated learning model to be trained according to the determined target task loss value;

[0037] Based on the first training deviation, adjusting the model parameters of the task-side underlying model of the vertical federated learning model to be trained;

[0038] The second training deviation is sent to the corresponding respective collaborative terminals, so that the corresponding respective collaborative terminals adjust the model parameters of the collaborative terminal underlying model at each collaborative terminal.

[0039] A fourth aspect of the present application provides a model training device, which is applied to a target collaboration terminal corresponding to a target task initiation terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiation terminals and at least one data providing terminal; the target task initiation terminal is any one of the multiple task initiation terminals; the target collaboration terminal is any one of the collaboration terminals of the target task initiation terminal; the collaboration terminal is selected from the multiple participating terminals; the device includes:

[0040] A collaboration terminal reading unit, configured to obtain a collaboration terminal task training sample of the target collaboration terminal;

[0041] The collaborative end training unit is used to repeatedly perform the following steps:

[0042] Inputting the collaborative end task training sample into the collaborative end underlying model of the longitudinal federated learning model to be trained, and obtaining second intermediate feature data output by the collaborative end underlying model;

[0043] The second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the target task is the task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by the target task initiating terminal inputting the initiating-end task training sample into the task-end underlying model of the longitudinal federated learning model to be trained;

[0044] In response to the instruction of the target task initiating terminal, the model parameters of the underlying model of the collaborative end are updated until the target task loss value converges to a preset target value, thereby obtaining a trained vertical federated learning model; the instruction is issued by the target task initiating terminal when determining the target task loss value based on the fused feature data, determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each of the collaborative terminals, and updating the model parameters of the task end underlying model and the upper-layer model.

[0045] In an optional embodiment, the collaboration terminal training unit is specifically configured to:

[0046] If an instruction sent by the target task initiating terminal is received, obtaining a second training deviation included in the instruction; the second training deviation is a deviation corresponding to the second intermediate feature data output by the collaborative terminal underlying model of the target collaborative terminal;

[0047] Based on the obtained second training deviation, the model parameters of the collaborative terminal underlying model of the target collaborative terminal are adjusted.

[0048] According to a fifth aspect provided by an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0049] According to the sixth aspect provided by an embodiment of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the method described in the first aspect or the second aspect is implemented.

[0050] The model training method in the above-mentioned embodiment of the present application is applied to the target task initiating terminal of the vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; by obtaining the initiating end task training sample; repeatedly performing the following steps: inputting the initiating end task training sample into the initiating end underlying model of the vertical federated learning model to be trained, and obtaining the first intermediate feature data output by the initiating end underlying model; receiving the second intermediate feature data sent by each collaborative terminal of the target task initiating terminal; the collaborative terminal is selected from the multiple participating terminals; the second intermediate feature data is the collaborative end underlying model of the vertical federated learning model to be trained of each collaborative terminal based on their own The intermediate feature data obtained from the collaborative end task training samples stored in the local storage; based on the first intermediate feature data and the second intermediate feature data, the fused feature data of the training sample of the target task is obtained; the target task is the task corresponding to the terminal initiating the target task; the fused feature data is input into the upper model of the longitudinal federated learning model to be trained to determine the target task loss value; the first training deviation of the first intermediate feature data and the second training deviation of each second intermediate feature data are determined respectively; based on the target task loss value and the first training deviation, the model parameters of the task end bottom model and the upper model are updated, and based on the second training deviation, each of the collaborative terminals is instructed to update the model parameters of the corresponding collaborative end bottom model, until the target task loss value converges to the preset target value, thereby obtaining a trained longitudinal federated learning model. Therefore, in this embodiment, multi-task parallel longitudinal federated learning can be realized, and multiple task initiating terminals can be trained in parallel, thereby improving the model training efficiency of the multi-task longitudinal federated learning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0052] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of a flow chart of obtaining a training sample set in a model training method provided in an embodiment of the present application;

[0054] Figure 3A flowchart of a process of obtaining fused feature data of a model training method provided by an embodiment of the present application is shown in FIG. 1.

[0055] Figure 4 A flowchart of updating model parameters of a model training method provided by an embodiment of the present application is shown in FIG. 2.

[0056] Figure 5 A flowchart of another model training method provided by an embodiment of the present application is shown in FIG. 3.

[0057] Figure 6 A flowchart of updating model parameters of a collaborative end bottom-layer model of another model training method provided by an embodiment of the present application is shown in FIG. 4.

[0058] Figure 7 A structural diagram of a model training device applied to a target task initiating terminal provided by an embodiment of the present application is shown in FIG. 5.

[0059] Figure 8 A structural diagram of another model training device applied to a target task initiating terminal provided by an embodiment of the present application is shown in FIG. 6.

[0060] Figure 9 A structural diagram of a model training device applied to a target collaborative terminal provided by an embodiment of the present application is shown in FIG. 7.

[0061] Figure 10 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 8. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0063] It should be noted that the terms "include" and "have" and their conjugations involved in the documents of the present application are intended to cover the inclusions not exclusively, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0064] Some words appearing in the text will be explained as follows:

[0065] (1) Federated Learning: Federated learning aims to build a federated learning model based on distributed datasets. In the process of model training, model-related information can be exchanged between parties (or in encrypted form), but the original data cannot. The exchange of model-related information does not expose any protected privacy of the data on each site.

[0066] (2) Vertical Federated Learning (VFL): In vertical federated learning, the data samples of different participants have a large overlap, but the overlap of sample features is not high. According to the different distribution of data used by federated learning at each participant, federated learning can be divided into three categories: horizontal federated learning (HFL), vertical federated learning, and federated transfer learning (FTL). For example, two companies provide different services to customers and have different aspects of customer data, but they have a large overlap in the customer groups they serve.

[0067] Recently, federated learning has gradually become the focus of research in the field of privacy-preserving computing. Federated learning allows data to be distributed across different devices and networks and is not visible to each other, solving security problems such as privacy leakage in data cooperation tasks. Vertical federated learning models heterogeneous data and is mainly applied to data cooperation between different organizations such as enterprises, governments, and institutions, and has a wide range of use backgrounds. In vertical federated learning, each participant has a common sample object, but the feature categories of the sample are different from each other, and only one participant provides task label information, which is called the task initiator, and the other participants are called data providers. Through vertical federated learning, the task initiator can use more comprehensive feature information of the sample to improve the accuracy of model prediction while ensuring data security.

[0068] In related technologies, vertical federated learning is usually single-task, including only one task initiator (Guest) and multiple data providers (Host). Among them, the task roles of the task initiator and the data provider are fixed, and the task initiator only uses labeled samples to participate in training. In actual scenarios, if the data provider also has business modeling needs and needs sample features of the task initiator, a new federated modeling task needs to be initiated again, which has a large time cost, and therefore the efficiency of the model training process in multi-task vertical federated learning is low.

[0069] To solve the existing technical problems, the embodiment of the present application provides a model training method applied to a target task initiation terminal of a vertical federated learning system; the vertical federated learning system comprises a plurality of participating terminals; the plurality of participating terminals comprise a plurality of task initiation terminals and at least one data providing terminal; the target task initiation terminal is any one of the plurality of task initiation terminals; an initiation terminal task training sample is obtained; the following steps are repeatedly executed: the initiation terminal task training sample is input into an initiation terminal bottom layer model of a vertical federated learning model to be trained, to obtain first intermediate feature data output by the initiation terminal bottom layer model; second intermediate feature data sent by each collaboration terminal of the target task initiation terminal is received; the collaboration terminal is selected from the plurality of participating terminals; the second intermediate feature data is intermediate feature data obtained by a collaboration terminal bottom layer model of a vertical federated learning model to be trained of each collaboration terminal based on a collaboration terminal task training sample stored locally by the collaboration terminal; fusion feature data of a training sample of a target task is obtained according to the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiation terminal; the fusion feature data is input into an upper layer model of the vertical federated learning model to be trained, to determine a target task loss value; a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data are respectively determined; model parameters of the task terminal bottom layer model and the upper layer model are updated based on the target task loss value and the first training deviation, and each collaboration terminal is instructed to update model parameters of the corresponding collaboration terminal bottom layer model based on the second training deviation, until the target task loss value converges to a preset target value, to obtain a trained vertical federated learning model. Thus, in the embodiment, the vertical federated learning of multiple tasks in parallel can be implemented, the training of the plurality of task initiation terminals in parallel can be performed, and the model training efficiency of the vertical federated learning process of multiple tasks is improved.

[0070] The model training method provided by the embodiment of the present application is applied to a vertical federated learning system, the vertical federated learning system comprises a plurality of participating terminals; the plurality of participating terminals comprise a plurality of task initiation terminals and at least one data providing terminal; and a target task initiation terminal is any one of the plurality of task initiation terminals. Each task initiation terminal can initiate a model training task and provide task label information corresponding to the model training task.

[0071] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the drawings.

[0072] The model training method provided by the embodiment of the present application is applied to a target task initiation terminal of a vertical federated learning system; the vertical federated learning system comprises a plurality of participating terminals; the plurality of participating terminals comprise a plurality of task initiation terminals and at least one data providing terminal; the target task initiation terminal is any one of the plurality of task initiation terminals; as Figure 1 shown, comprising the following steps:

[0073] Step S101: Obtain an initiator task training sample.

[0074] During specific implementation, the target task initiating terminal of the vertical federated learning system obtains the initiating task training samples.

[0075] In the embodiment of the present application, there are multiple task initiating terminals, for example, the number of task initiating terminals may be 2, 3, 4, ..., Q. For a clearer description, the following embodiment is described by taking the case where the number of task initiating terminals is 2 as an example.

[0076] It should be noted that the step of obtaining the initiating end task training samples in step S101 may be to obtain the training samples of each wholesale initiating end task one by one.

[0077] For example, assume that there are three participating terminals a, b, and c, of which participating terminal a and participating terminal c are task initiating terminals. Participating terminal a and participating terminal c may also be referred to as task initiating terminal a and task initiating terminal c, respectively. Task initiating terminal a and task initiating terminal c respectively undertake tasks job1 and job2, and participating terminal b is a data providing terminal. The following embodiments all take the target task initiating terminal of the vertical federated learning system as the task initiating terminal a as an example to illustrate the model training terminal method applied to the target task initiating terminal. Task initiating terminal a obtains the initiating terminal task training sample Among them, x a,1,i represents the i-th batch of feature data provided by participating terminal a to job1, x a,2,i It represents the i-th batch of feature data provided by participating terminal a to job2.

[0078] In an optional embodiment, before obtaining the initiator task training sample in step S101, Figure 2 As shown, the method further includes the following steps:

[0079] Step S201 : Selecting a first preset number of participating terminals from participating terminals other than the target task initiating terminal as cooperative terminals of the target task initiating terminal.

[0080] In specific implementation, before obtaining the initiating terminal task training sample in step S101, data preparation is performed first. The above data preparation process includes: all task initiating terminals select cooperative terminals.

[0081] For example, assume that in a multi-task vertical federated learning scenario, there are p participating terminals, q of which are task-initiating terminals, and the remaining pq terminals serve only as data providers. Each task-initiating terminal selects k participating terminals from the remaining p-1 participating terminals as collaborative terminals for its task.

[0082] Step S202: determine the intersection samples with each collaborative terminal through private intersection to obtain the training sample set of the target task; the training sample set includes the initiating terminal task training samples stored in the target task initiating terminal and the collaborative terminal task training samples stored in each collaborative terminal of the target task initiating terminal.

[0083] In specific implementation, the data preparation process also includes all task-initiating terminals determining the sample data set for each task through privacy intersection with their corresponding collaborative terminals, and dividing the sample set of each task according to the set batch size to ensure that each training batch contains sample data of each task in equal proportion.

[0084] For example, each task initiating terminal privately intersects with the selected k collaborative terminals, and the intersection samples are used as the training sample set. Each participant maintains a training sample table, which can be shown in Table 1 below:

[0085] Table 1

[0086] Task number Sample ID collection job 1 <![CDATA[[id 1,1 ,id 1,2 ,...,id 1,n1 , job 2 [[id 2,1 ,id 2,2 ,...,id 2,n2 ]]]> … … job q <![CDATA[[id q,1 ,id q,2 ,...,id 3,nq ,

[0087] As shown in Table 1, in multi-task vertical federated learning, the total number of samples in the training sample set can be N = n1 + n2 + ... + n q In some embodiments, the training batch size can be set to B, and the batch size of each task can be set to Each participating terminal of each task divides the training data set according to the batch size of the task. The number of batches divided into which the training sample set of each task is divided is the same, so that each batch of training samples will contain sample data of each task in equal proportion.

[0088] In step S101, after obtaining the initiator task training sample, the following steps are repeated.

[0089] Step S102: Input the initiator task training sample into the initiator underlying model of the longitudinal federated learning model to be trained to obtain the first intermediate feature data output by the initiator underlying model.

[0090] In the embodiment of the present application, in the vertical federated learning system, the model of the task initiating terminal mainly consists of two parts: the task-side underlying model and the upper-layer model. Since the data providing terminal has no business modeling requirements, it only includes the collaboration-side underlying model. Among them, the task-side underlying model and the collaboration-side underlying model are underlying models of the same structure. The underlying model in the embodiment of the present application is responsible for the feature conversion of the original input training samples; the upper-layer model is responsible for fitting its own task objectives. In this vertical federated learning system, the underlying model is shared between tasks, that is, if a participating terminal corresponds to multiple model training tasks, for example, the participating terminal POT01 corresponds to the model training task Task_1 and the model training task Task_2, then the model training task Task_1 and the model training task Task_2 share the underlying model of the participating terminal POT01.

[0091] For example, assume three participating terminals a, b, and c, where participating terminal a and participating terminal c are task initiating terminals and participating terminal b is a data providing terminal. Participating terminal a and participating terminal c are respectively referred to as task initiating terminal a and task initiating terminal c. The underlying model of task initiating terminal a, participating terminal b, and task initiating terminal c is represented as BM a ,BM b ,BM c , the upper layer models of task initiation terminal a and task initiation terminal c are represented as TM a ,TM c By initiating the task training sample Input the initiator underlying model BM of the vertical federated learning model to be trained a , get the underlying model BM of the initiator a Output first intermediate feature data in,

[0092]

[0093] Other participants, such as the task initiating terminal c, are similar to the task initiating terminal a.

[0094] Step S103: receiving second intermediate feature data sent by each cooperative terminal of the target task initiating terminal.

[0095] The collaborative terminal is selected from a plurality of participating terminals. The second intermediate feature data is intermediate feature data obtained by the collaborative terminal underlying model of the longitudinal federated learning model to be trained of each collaborative terminal based on the collaborative terminal task training samples stored locally.

[0096] Exemplarily, the cooperative terminal is selected from a plurality of participating terminals. Assume that the cooperative terminals of the task initiating terminal a are participating terminals b and c. The task initiating terminal a receives the second intermediate feature data sent by each cooperative terminal of the task initiating terminal a, wherein the received second intermediate feature data may include the second intermediate feature data z sent by the participating terminal b. b,1,i , and the second intermediate feature data z sent by the participating terminal c c,1,i .

[0097] Step S104: obtaining fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data; the target task is the task corresponding to the target task initiating terminal.

[0098] In an embodiment of the present application, the task initiator merges the intermediate feature data.

[0099] In an optional embodiment, the process of obtaining the fused feature data of the training sample of the target task according to the first intermediate feature data and the second intermediate feature data is as follows: Figure 3 As shown, this is achieved through the following steps:

[0100] Step S301: Determine the sample object identifiers corresponding to the first intermediate feature data and each second intermediate feature data respectively.

[0101] In step S302 , the first intermediate feature data and each second intermediate feature data corresponding to the same sample object identifier are concatenated in a preset order to obtain fused feature data of the training sample of the target task.

[0102] For example, the task initiating terminal a uses the first intermediate feature data The second intermediate feature data z sent by the participating terminal b b,1,i , and the second intermediate feature data z sent by the participating terminal c c,1,i , get the fusion feature data of the training sample of the target task Task_targ [Z a,1,i ,Z b,1,i ,Z c,1,i ].

[0103] Step S105: Input the fused feature data into the upper model of the longitudinal federated learning model to be trained to determine the target task loss value.

[0104] In specific implementation, the fused feature data is input into the upper model of the longitudinal federated learning model to be trained, and the predicted value is calculated to determine the target task loss value.

[0105] For example, the task initiating terminal a fuses the feature data [Z a,1,i ,Z b,1,i,Z c,1,i ] Input to the upper model TM of the longitudinal federated learning model to be trained a , determine the target task loss value L 1,i .

[0106] Y′ 1,i =TM a ([z a,1,i z b,1,i z c,1,i ]),

[0107] L 1,i =loss(Y 1,i ,Y′ 1,i ),

[0108] Among them, Y′ 1,i is the predicted value of the fused feature data;

[0109] Y 1,i is the label value of the fused feature data;

[0110] loss(,) is the loss function.

[0111] Step S106 , determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data respectively.

[0112] During specific implementation, the target task initiating terminal determines the upper-layer model gradient of the target task initiating terminal based on the target task loss value and the fused feature data, and then calculates the first training deviation of the first intermediate feature data and the second training deviation of each second intermediate feature data based on the determined upper-layer model gradient and the predicted value of the fused feature data.

[0113] For example, the first training deviation of the first intermediate feature data and the second training deviation of each second intermediate feature data can be calculated using the following formula:

[0114]

[0115]

[0116] Among them, ΔTM a is the upper model gradient of the target task initiating terminal;

[0117] Δz a,1,i is the first intermediate feature data z a,1,i The first training deviation of

[0118] Δz b,1,i is the second intermediate feature data z b,1,i The second training deviation;

[0119] Δz c,1,i is the second intermediate feature data z c,1,i The second training deviation.

[0120] Step S107: Update the model parameters of the task-side underlying model and the upper-layer model based on the target task loss value and the first training deviation, and instruct each collaborative terminal to update the model parameters of its corresponding collaborative-side underlying model based on the second training deviation until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

[0121] In an optional embodiment, in step S107, the model parameters of the task-side bottom model and the upper model are updated based on the target task loss value and the first training deviation, and each collaborative terminal is instructed to update the model parameters of the corresponding collaborative terminal bottom model based on the second training deviation, such as Figure 4 As shown, this is achieved through the following steps:

[0122] Step S401: Adjust the model parameters of the upper model of the vertical federated learning model to be trained according to the determined target task loss value.

[0123] In specific implementation, the process of adjusting the model parameters of the upper model of the longitudinal federated learning model to be trained according to the determined target task loss value can be to adjust the model parameters of the upper model of the longitudinal federated learning model to be trained according to the upper model gradient of the target task initiating terminal corresponding to the determined target task loss value.

[0124] Step S402: Based on the first training deviation, adjust the model parameters of the task-side underlying model of the vertical federated learning model to be trained.

[0125] Step S403: sending the second training deviation to the corresponding collaborative terminals, so that the corresponding collaborative terminals adjust the model parameters of the collaborative terminal bottom layer model at each collaborative terminal respectively.

[0126] In an embodiment of the present application, all participating terminals convert input features through the underlying model and output intermediate feature data. All participating terminals send this intermediate feature data to the corresponding task initiating terminal based on the task to which they belong. All task initiating terminals then sequentially concatenate the intermediate feature data, calculate sample prediction values ​​and losses, and then update the upper-level model and calculate the training deviation of the intermediate data. Next, all task initiating terminals return the training deviation to the data providing terminal to which they belong. All participating terminals then calculate the parameter gradients of the underlying model based on the training deviation and task weights, completing the update.

[0127] For example, the task initiating terminal a sets the second training deviation Δz b,1,i and Δz c,1,i, and sent to the corresponding collaborative terminals, so that the corresponding collaborative terminals can adjust the model parameters of the collaborative terminal underlying model at each collaborative terminal respectively.

[0128] In some embodiments of the present application, the task weights of each task in the multi-task can also be pre-set. Since the training difficulty and sample set size of each task are different, the hyperparameter task weight λ is introduced. The task weight determines the degree of influence of each task on the update of the underlying model. Assume that the task weights of job 1 and job 2 are λ1 and λ2 respectively, the number of training samples are n1 and n2, and the task initiating terminal a also provides the data of the collaborative end task training sample corresponding to job 2 to the task initiating terminal c; and it has been obtained by derivation of the upper model and The training bias is and Then the gradient ΔBM of the underlying model of the task initiating terminal a is a The calculation can be:

[0129]

[0130] Among them, λ1+λ2=1;

[0131] The first intermediate feature data z of the task initiating terminal a a,1 The first training deviation of

[0132] The second training deviation of the collaborative end task training sample corresponding to job 2 provided by the task initiating terminal a to the task initiating terminal c;

[0133] x a,1 The first intermediate feature data of the task initiating terminal a Corresponding initiator task training samples;

[0134] x a,2 The second intermediate feature data provided by the task initiating terminal a to the task initiating terminal c Corresponding collaborative end task training samples.

[0135] In some embodiments of the present application, after the target task loss value of the target task initiating terminal converges to a preset target value, the further embodiment includes: the target task initiating terminal sends a stop instruction to each cooperative terminal of the target task initiating terminal, so that each cooperative terminal of the target task initiating terminal archives the current model information respectively according to the stop instruction, and reduces the task weight of the target task corresponding to the target task initiating terminal to 0. The above embodiment provides an early stopping mechanism. During the multi-task training process, the convergence speed of each task is different, so each task cannot reach the convergence state at the same time. Through the above early stopping mechanism, the converged target task can be effectively prevented from overfitting, and the training speed of the non-converged tasks can be accelerated.

[0136] The model training method provided in the embodiment of the present application is applied to the target task initiating terminal of the vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; including: obtaining an initiating end task training sample; repeating the following steps: inputting the initiating end task training sample into the initiating end underlying model to obtain first intermediate feature data; receiving second intermediate feature data sent by each collaborative terminal; obtaining fused feature data based on the first intermediate feature data and the second intermediate feature data; inputting the fused feature data into the upper model of the vertical federated learning model to be trained, determining the target task loss value; respectively determining the first training deviation of the first intermediate feature data and the second training deviation of each second intermediate feature data; updating the model parameters of the task end underlying model and the upper model based on the target task loss value and the first training deviation, and instructing each collaborative terminal to update the model parameters of the corresponding collaborative end underlying model based on the second training deviation until the target task loss value converges to a preset target value. Thus, in this embodiment, multi-task parallel vertical federated learning can be realized, and multiple task initiating terminals can be trained in parallel, thereby improving the model training efficiency of the multi-task vertical federated learning process.

[0137] The embodiment of the present application provides another model training method, which is applied to a target collaborative terminal corresponding to a target task initiating terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the target collaborative terminal is any one of the collaborative terminals of the target task initiating terminal; the collaborative terminal is selected from the multiple participating terminals; Figure 5 As shown, the following steps are included:

[0138] Step S501: Acquire a collaborative terminal task training sample of a target collaborative terminal.

[0139] After step S501, the following steps are repeatedly executed.

[0140] In step S502, the collaborative-end task training sample is input into the collaborative-end underlying model of the longitudinal federated learning model to be trained, to obtain second intermediate feature data output by the collaborative-end underlying model.

[0141] In step S503, the second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fusion feature data of the training sample of the target task according to the first intermediate feature data and the second intermediate feature data sent by each collaborative terminal of the target task initiating terminal; the target task is a task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by inputting the initiating-end task training sample into the task-end underlying model of the longitudinal federated learning model to be trained by the target task initiating terminal.

[0142] In step S504, the model parameters of the collaborative-end underlying model are updated in response to the indication of the target task initiating terminal until the target task loss value converges to a preset target value, to obtain the trained longitudinal federated learning model; the indication is issued by the target task initiating terminal when determining the target task loss value according to the fusion feature data, respectively determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each collaborative terminal, and updating the model parameters of the task-end underlying model and the upper model.

[0143] In an optional embodiment, the process of updating the model parameters of the collaborative-end underlying model in response to the indication of the target task initiating terminal is implemented by the following steps as shown in Figure 6

[0144] In step S601, if the indication sent by the target task initiating terminal is received, the second training deviation included in the indication is obtained; the second training deviation is a deviation corresponding to the second intermediate feature data output by the collaborative-end underlying model of the target collaborative terminal.

[0145] In step S602, the model parameters of the collaborative-end underlying model of the target collaborative terminal are adjusted based on the obtained second training deviation.

[0146] Based on the same inventive concept as the above model training method, a model training device is also provided in the embodiments of the present application. Since the device is the device corresponding to the model training method of the present application, and the principles of solving problems of the device are similar to those of the method, the implementation of the device can be referred to the implementation of the above method, and the repeated parts will not be repeated.

[0147] Figure 7 ​A schematic diagram of the structure of a model training device provided in an embodiment of the present application is shown, wherein the model training device is applied to a target task initiating terminal of a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; Figure 7 As shown, the model training device includes a task-side reading unit 701 and a task-side training unit 702.

[0148] The task-side reading unit 701 is used to obtain the initiator-side task training sample;

[0149] The task-side training unit 702 is configured to repeatedly perform the following steps:

[0150] Inputting the initiator task training sample into the initiator bottom layer model of the longitudinal federated learning model to be trained, and obtaining the first intermediate feature data output by the initiator bottom layer model;

[0151] Receiving second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the cooperating terminal is selected from multiple participating terminals; the second intermediate feature data is intermediate feature data obtained by the cooperating terminal underlying model of the longitudinal federated learning model to be trained of each cooperating terminal based on the cooperating terminal task training samples stored locally;

[0152] Obtaining fused feature data of a training sample of a target task based on the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiating terminal;

[0153] Input the fused feature data into the upper model of the longitudinal federated learning model to be trained to determine the target task loss value;

[0154] respectively determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data;

[0155] Based on the target task loss value and the first training deviation, the model parameters of the task-side underlying model and the upper-level model are updated, and based on the second training deviation, each collaborative terminal is instructed to update the model parameters of its corresponding collaborative-side underlying model until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

[0156] In an optional embodiment, the task-side training unit 702 is specifically configured to:

[0157] respectively determining the sample object identifiers corresponding to the first intermediate feature data and each second intermediate feature data;

[0158] The first intermediate feature data and each second intermediate feature data corresponding to the same sample object identifier are respectively spliced ​​in a preset order to obtain the fused feature data of the training sample of the target task.

[0159] In an optional embodiment, as Figure 8 As shown, the device further includes a sample setting unit 801, which is used to:

[0160] Selecting a first preset number of participating terminals from participating terminals other than the target task initiating terminal as cooperative terminals of the target task initiating terminal;

[0161] The intersection samples are determined by private intersection with each collaborative terminal to obtain the training sample set of the target task; the training sample set includes the initiating terminal task training samples stored in the target task initiating terminal and the collaborative terminal task training samples stored in each collaborative terminal of the target task initiating terminal.

[0162] In an optional embodiment, the task-side training unit 702 is specifically configured to:

[0163] Adjust the model parameters of the upper model of the longitudinal federated learning model to be trained according to the determined target task loss value;

[0164] Based on the first training deviation, the model parameters of the task-side underlying model of the vertical federated learning model to be trained are adjusted;

[0165] The second training deviation is sent to the corresponding cooperative terminals, so that the corresponding cooperative terminals respectively adjust the model parameters of the cooperative terminal bottom layer model at each cooperative terminal.

[0166] Figure 9 A structural schematic diagram of another model training device provided in an embodiment of the present application is shown, wherein the model training device is applied to a target collaboration terminal corresponding to a target task initiation terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiation terminals and at least one data providing terminal; the target task initiation terminal is any one of the multiple task initiation terminals; the target collaboration terminal is any one of the collaboration terminals of the target task initiation terminal; the collaboration terminal is selected from the multiple participating terminals; Figure 9 As shown, the model training device includes a collaboration end reading unit 901 and a collaboration end training unit 902.

[0167] The collaboration terminal reading unit 901 is used to obtain the collaboration terminal task training sample of the target collaboration terminal;

[0168] The cooperation terminal training unit 902 is configured to repeatedly perform the following steps:

[0169] Input the collaborative end task training sample into the collaborative end underlying model of the longitudinal federated learning model to be trained, and obtain the second intermediate feature data output by the collaborative end underlying model;

[0170] The second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the target task is the task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by the target task initiating terminal inputting the initiating terminal task training sample into the task-side underlying model of the longitudinal federated learning model to be trained;

[0171] In response to the instruction of the target task initiating terminal, the model parameters of the underlying model of the collaborative end are updated until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model; the instruction is issued by the target task initiating terminal when determining the target task loss value based on the fused feature data, determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each collaborative terminal, and updating the model parameters of the underlying model and the upper-layer model of the task end.

[0172] In an optional embodiment, the cooperation terminal training unit 902 is specifically configured to:

[0173] If an instruction sent by the target task initiating terminal is received, a second training deviation included in the instruction is obtained; the second training deviation is a deviation corresponding to the second intermediate feature data output by the collaborative terminal underlying model of the target collaborative terminal;

[0174] Based on the obtained second training deviation, the model parameters of the collaborative terminal underlying model of the target collaborative terminal are adjusted.

[0175] Based on the same inventive concept as the above method embodiment, an electronic device is also provided in the embodiment of the present application. The electronic device can be used for model training. In one embodiment, the electronic device can be a task initiating terminal. In another embodiment, the electronic device can also be a cooperative terminal of the task initiating terminal. In this embodiment, the structure of the electronic device can be as follows: Figure 10 As shown, it includes a memory 1001 , a communication module 1003 and one or more processors 1002 .

[0176] Memory 1001 is used to store computer programs executed by processor 1002. Memory 1001 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and programs required for running instant messaging functions, while the data storage area may store various instant messaging messages and operating instruction sets.

[0177] Memory 1001 may be a volatile memory, such as random-access memory (RAM); a non-volatile memory, such as read-only memory, flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1001 may be a combination of the above memories.

[0178] The processor 1002 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 1002 is configured to implement the above-mentioned model training method when calling the computer program stored in the memory 1001 .

[0179] The communication module 1003 is used to communicate with terminal devices and other servers.

[0180] The specific connection medium between the memory 1001, the communication module 1003 and the processor 1002 is not limited in the embodiment of the present application. Figure 10 In the embodiment, the memory 1001 and the processor 1002 are connected via a bus 1004. The bus 1004 is connected to the processor 1002 via a bus 1004. Figure 10 The connections between the other components are shown in bold lines, which are only for illustration and are not intended to be limiting. The bus 1004 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0181] According to one aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the model training method in the above embodiment. The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0182] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A model training method, characterized in that: A target task initiating terminal applied to a vertical federated learning system; the vertical federated learning system includes a plurality of participating terminals; the plurality of participating terminals include a plurality of task initiating terminals and at least one data providing terminal; The target task initiating terminal is any one of the multiple task initiating terminals; the method includes: Obtain initiator task training samples; Repeat the following steps: Inputting the initiator task training sample into the initiator underlying model of the longitudinal federated learning model to be trained, and obtaining first intermediate feature data output by the initiator underlying model; Receiving second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the cooperating terminal is selected from the multiple participating terminals; the second intermediate feature data is intermediate feature data obtained by the cooperating terminal underlying model of the longitudinal federated learning model to be trained of each cooperating terminal based on the cooperating terminal task training samples stored locally; Obtaining fused feature data of a training sample of a target task based on the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiating terminal; Inputting the fused feature data into an upper model of the longitudinal federated learning model to be trained to determine a target task loss value; respectively determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data; Based on the target task loss value and the first training deviation, the model parameters of the task-side underlying model and the upper-layer model are updated, and based on the second training deviation, each of the collaborative terminals is instructed to update the model parameters of the corresponding collaborative-side underlying model until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

2. The method according to claim 1, characterized in that The obtaining, based on the first intermediate feature data and the second intermediate feature data, fused feature data of the training sample of the target task includes: respectively determining the sample object identifiers corresponding to the first intermediate feature data and each second intermediate feature data; The first intermediate feature data and each second intermediate feature data corresponding to the same sample object identifier are respectively spliced ​​in a preset order to obtain fused feature data of the training sample of the target task.

3. The method according to claim 1, characterized in that Before obtaining the initiator task training sample, the method further includes: Selecting a first preset number of participating terminals from participating terminals other than the target task initiating terminal as cooperative terminals of the target task initiating terminal; The intersection samples are determined by private intersection with each of the collaborative terminals to obtain a training sample set for the target task; the training sample set includes the initiating end task training samples stored in the target task initiating terminal, and the collaborative end task training samples respectively stored in each of the collaborative terminals of the target task initiating terminal.

4. The method according to claim 1, wherein The updating of the model parameters of the task-side underlying model and the upper-layer model based on the target task loss value and the first training deviation, and instructing each of the collaborative terminals to update the model parameters of the corresponding collaborative terminal underlying model based on the second training deviation, includes: Adjust the model parameters of the upper model of the longitudinal federated learning model to be trained according to the determined target task loss value; Based on the first training deviation, adjusting the model parameters of the task-side underlying model of the vertical federated learning model to be trained; The second training deviation is sent to the corresponding respective collaborative terminals, so that the corresponding respective collaborative terminals adjust the model parameters of the collaborative terminal underlying model at each collaborative terminal.

5. A model training method, characterized in that: A target collaborative terminal corresponding to a target task initiating terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the target collaborative terminal is any one of the collaborative terminals of the target task initiating terminal; The cooperative terminal is selected from the plurality of participating terminals; and the method includes: Obtaining a collaborative terminal task training sample of the target collaborative terminal; Repeat the following steps: Inputting the collaborative end task training sample into the collaborative end underlying model of the longitudinal federated learning model to be trained, and obtaining second intermediate feature data output by the collaborative end underlying model; The second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the target task is the task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by the target task initiating terminal inputting the initiating-end task training sample into the task-end underlying model of the longitudinal federated learning model to be trained; In response to the instruction of the target task initiating terminal, the model parameters of the underlying model of the collaborative end are updated until the target task loss value converges to a preset target value, thereby obtaining a trained longitudinal federated learning model; the instruction is issued by the target task initiating terminal when determining the target task loss value based on the fused feature data, respectively determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each of the collaborative terminals, and updating the model parameters of the underlying model and the upper-layer model of the task end.

6. The method according to claim 5, characterized in that The updating of the model parameters of the underlying model of the collaboration end in response to the instruction of the target task initiating terminal includes: If an instruction sent by the target task initiating terminal is received, obtaining a second training deviation included in the instruction; the second training deviation is a deviation corresponding to the second intermediate feature data output by the collaborative terminal underlying model of the target collaborative terminal; Based on the obtained second training deviation, the model parameters of the collaborative terminal underlying model of the target collaborative terminal are adjusted.

7. A model training device, characterized in that: A target task initiating terminal applied to a vertical federated learning system; the vertical federated learning system includes a plurality of participating terminals; the plurality of participating terminals include a plurality of task initiating terminals and at least one data providing terminal; The target task initiating terminal is any one of the multiple task initiating terminals; the device includes: The task-side reading unit is used to obtain the initiator-side task training samples; The task-side training unit is used to repeatedly perform the following steps: Inputting the initiator task training sample into the initiator underlying model of the longitudinal federated learning model to be trained, and obtaining first intermediate feature data output by the initiator underlying model; Receiving second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the cooperating terminal is selected from the multiple participating terminals; the second intermediate feature data is intermediate feature data obtained by the cooperating terminal underlying model of the longitudinal federated learning model to be trained of each cooperating terminal based on the cooperating terminal task training samples stored locally; Obtaining fused feature data of a training sample of a target task based on the first intermediate feature data and the second intermediate feature data; the target task is a task corresponding to the target task initiating terminal; Inputting the fused feature data into an upper model of the longitudinal federated learning model to be trained to determine a target task loss value; respectively determining a first training deviation of the first intermediate feature data and a second training deviation of each second intermediate feature data; Based on the target task loss value and the first training deviation, the model parameters of the task-side underlying model and the upper-layer model are updated, and based on the second training deviation, each of the collaborative terminals is instructed to update the model parameters of the corresponding collaborative-side underlying model until the target task loss value converges to the preset target value, thereby obtaining a trained vertical federated learning model.

8. A model training device, characterized in that: A target collaborative terminal corresponding to a target task initiating terminal in a vertical federated learning system; the vertical federated learning system includes multiple participating terminals; the multiple participating terminals include multiple task initiating terminals and at least one data providing terminal; the target task initiating terminal is any one of the multiple task initiating terminals; the target collaborative terminal is any one of the collaborative terminals of the target task initiating terminal; The cooperative terminal is selected from the plurality of participating terminals; the apparatus comprises: A collaboration terminal reading unit, configured to obtain a collaboration terminal task training sample of the target collaboration terminal; The collaborative end training unit is used to repeatedly perform the following steps: Inputting the collaborative end task training sample into the collaborative end underlying model of the longitudinal federated learning model to be trained, and obtaining second intermediate feature data output by the collaborative end underlying model; The second intermediate feature data is sent to the target task initiating terminal, so that the target task initiating terminal obtains fused feature data of the training sample of the target task based on the first intermediate feature data and the second intermediate feature data sent by each cooperating terminal of the target task initiating terminal; the target task is the task corresponding to the target task initiating terminal; the first intermediate feature data is obtained by the target task initiating terminal inputting the initiating-end task training sample into the task-end underlying model of the longitudinal federated learning model to be trained; In response to the instruction of the target task initiating terminal, the model parameters of the underlying model of the collaborative end are updated until the target task loss value converges to a preset target value, thereby obtaining a trained longitudinal federated learning model; the instruction is issued by the target task initiating terminal when determining the target task loss value based on the fused feature data, respectively determining the first training deviation of the first intermediate feature data and the second training deviation of the second intermediate feature data sent by each of the collaborative terminals, and updating the model parameters of the underlying model and the upper-layer model of the task end.

9. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 or claims 5 to 6 is implemented.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the processor implements the method according to any one of claims 1 to 4 or claims 5 to 6.

Images