Longitudinal federal learning method and system based on virtual feature generation

By introducing virtual feature data and virtual external features into vertical federated learning, it transforms it into horizontal federated learning paradigm, and solves the problems of low communication efficiency and strict model input features of vertical federated learning, and realizes the support of multi-layer nonlinear computing, improving the improvement of Communication efficiency and model performance are achieved.

CN119089979APending Publication Date: 2024-12-06CHINA CITIC BANK CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411023483.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Communication efficiency is low in vertical federated learning. The existing solutions are mainly aimed at horizontal federated learning. They cannot effectively support multi-layer nonlinear computing, and have strict requirements on model input features and only support single-layer neural networks.

Method used

By introducing virtual feature data, all participants have full feature data locally, vertical federated learning is transformed into horizontal federated learning paradigm, and task models are trained on the client side based on virtual feature generation, and aggregation and fine-tuning are carried out on the server side. Global task model to support multi-layer nonlinear computing.

Benefits of technology

It significantly improves the communication efficiency of vertical federated learning, supports multi-layer nonlinear computing on the client, solves the problem that the model input features must be decomposed into linear combinations, and improves the model's expression ability and generalization performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089979A_ABST
    Figure CN119089979A_ABST
Patent Text Reader

Abstract

The invention relates to a longitudinal federated learning method and system based on virtual feature generation. The method comprises the following steps: initializing a task model based on longitudinal federated learning; training the task model at the client to obtain a first task model; aggregating the first task model at the server side to obtain a global task model; performing fine adjustment on the global task model according to the input features; and repeating the steps to generate a final task model. According to the method, the communication frequency is reduced by increasing the local calculation amount, so that the problem of low communication efficiency of longitudinal federated learning is solved, and the communication efficiency of longitudinal federated learning is improved; virtual feature data is introduced, so that participants have full-amount feature data locally, the problems that model input is required and only a single-layer neural network is supported are solved, and multi-layer nonlinear calculation of a client can be supported.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a longitudinal federated learning method and system based on virtual feature generation. Background Art

[0002] In the training process of vertical federated learning, multiple parties need to exchange gradient update information for joint calculations at each iteration, so they face a severe communication efficiency bottleneck. In actual use, they can only support simple linear models or tree models, and have no good support for more complex calculations such as neural networks. At present, relevant work has explored improving communication efficiency, but most of them are based on horizontal federated learning. The main representative work is the FedAvg scheme and its related improvements. For vertical federated learning, the main schemes are FedBCD.

[0003] FedAvg solution: In horizontal federated learning based on sample segmentation, by running stochastic gradient descent in parallel to perform multiple local updates, it is possible to use local data to train the global model while protecting privacy data, reduce communication overhead and support distributed devices, while improving the accuracy and generalization performance of the model. The specific steps are as follows:

[0004] (1) The server initializes the global model parameter w_0;

[0005] (2) All local devices randomly select a portion of the data set and train the model parameters w_j locally;

[0006] (3) All local devices upload local model parameters w_j to the server;

[0007] (4) The server calculates the weighted average of all local model parameters and broadcasts it to all local devices;

[0008] (5) All local devices use the weighted average value as the processing of local model parameters, and repeat 2-4 until the global model converges.

[0009] FedBCD scheme: For vertical federated learning schemes, the FedBCD scheme reduces the number of communication rounds by requiring participants to perform multiple rounds of updates locally before communicating. The specific steps are as follows:

[0010] (1) Perform multiple rounds of gradient updates locally;

[0011] (2) Send the intermediate parameter wx to the last participant K;

[0012] (3) Participant K combines all intermediate parameters to obtain the overall gradient and sends it to each participant separately;

[0013] (4) Each participant receives the overall gradient and updates its local gradient.

[0014] With the vigorous development of artificial intelligence, especially deep learning technology, communication efficiency has become the main bottleneck that determines the widespread application of vertical federated learning. Most solutions, including FedAvg and its subsequent improvements, are oriented towards horizontal federated learning, while in actual applications, vertical federation has a wider range of application scenarios. Although FedBCD is aimed at vertical federated learning, it allows each participant to exchange information after multiple local calculations to reduce communication overhead, but the solution requires that the model input features can be decomposed into a linear combination of the features of each client, and only supports single-layer neural networks, and cannot perform multi-layer nonlinear calculations on features of different dimensions of multiple clients. Summary of the invention

[0015] In order to address the shortcomings of the prior art, the present invention provides a vertical federated learning method and system based on virtual feature generation, which reduces the number of communications by increasing the local computing amount to solve the problem of low communication efficiency of vertical federated learning. The introduction of virtual feature data enables each participant to have the full feature data locally, and transforms vertical federated learning into a horizontal federated learning paradigm, which solves the problem that the model input features must be decomposable into linear combinations of the features of each client, and can support multi-layer nonlinear calculations on the client.

[0016] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0017] A longitudinal federated learning method based on virtual feature generation, characterized by comprising:

[0018] S1. Initialize the task model based on vertical federated learning;

[0019] S2. Train the task model on the client to obtain a first task model, wherein the task model is used to accept all input features and output a predicted label, wherein the input features include local real features and external real features;

[0020] S3. Aggregate the first task model on the server to obtain a global task model;

[0021] S4. fine-tuning the global task model according to the input features;

[0022] S5. Repeat steps S2 to S4 to generate the final task model.

[0023] Furthermore, before the method, it also includes: deploying N federated learning computing nodes, N is greater than 2, wherein 1 of the federated learning computing nodes is a server node, and N-1 of the federated learning computing nodes are client nodes.

[0024] Furthermore, the step of training the task model on the client to obtain a first task model includes:

[0025] Setting the expected label, learning the conditional generative model on the client, inputting the local real feature and the expected label, and outputting the virtual external feature;

[0026] The task model is trained according to the local real features and the virtual external features to obtain a first task model.

[0027] Furthermore, the method further comprises:

[0028] Obtain a first prediction label according to the first task model;

[0029] Obtaining a task loss function based on the expected label and the first predicted label prediction loss;

[0030] The first task model and the conditional generation model are optimized using the task loss function as an objective function.

[0031] Furthermore, after the server aggregates the first task model to obtain a global task model, the method further includes:

[0032] The global task model is distributed to clients.

[0033] Before fine-tuning the global task model according to the input features, the method further includes:

[0034] After receiving the global task model, the clients communicate with each other.

[0035] Further, fine-tuning the global task model according to the input features includes:

[0036] Inputting the input feature into the global task model to obtain a second prediction label;

[0037] The global task model is optimized according to the prediction loss of the true label and the second predicted label, wherein the true label is directly obtained from the data set.

[0038] The present invention also relates to a longitudinal federated learning system based on virtual feature generation, characterized in that it includes:

[0039] Initialization module, used to initialize the task model based on vertical federated learning;

[0040] A training module, used for training the task model on the client to obtain a first task model, wherein the task model is used for accepting all input features and outputting a predicted label, wherein the input features include local real features and external real features;

[0041] An aggregation module, used for aggregating the first task models on the server side to obtain a global task model;

[0042] A fine-tuning module, used for fine-tuning the global task model according to the input features;

[0043] The final generation module is used to repeat the steps to generate the final task model.

[0044] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0045] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0046] The memory is used to store program instructions;

[0047] The processor is used to execute the above method by calling program instructions.

[0048] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0049] The beneficial effects of the present invention are:

[0050] The present invention aims to improve the communication efficiency of vertical federated learning to solve the current problems of low communication efficiency, model input requirements and only support for single-layer neural networks in vertical federated learning. The method and system for vertical federated learning based on virtual feature generation described in the present invention are used to reduce the number of communications by increasing the amount of local computing, thereby solving the problem of low communication efficiency of vertical federated learning; the introduction of virtual feature data allows each participant to have the full amount of feature data locally, so that vertical federated learning can be transformed into a horizontal federated learning paradigm, solving the problem that the model input features must be decomposable into a linear combination of the features of each client, and supporting multi-layer nonlinear calculations on the client. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0052] Figure 1 A schematic diagram of a flow chart of a longitudinal federated learning method based on virtual feature generation according to the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a vertical federated learning system based on virtual feature generation according to the present invention;

[0054] Figure 3 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments, and these embodiments are only used to illustrate the present application and are not used to limit the scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0056] Vertical federated learning iterates and optimizes the model through multiple rounds of communication between the client and the server. Each round of communication includes four parts: (1) each client obtains the public key from the server; (2) the intermediate results are encrypted and exchanged between the clients, and the intermediate results are used to help calculate the gradient and loss value; (3) the client sends the encrypted result to the server; (4) the server sends the decrypted result to each client. However, the existing vertical federated learning requires repeated network communication between clients, which is inefficient. If the number of communications between clients can be reduced and more iterative optimization can be performed locally, the communication efficiency can be significantly improved. Therefore, a vertical federated learning method and system based on virtual feature generation is proposed.

[0057] like Figure 1 As shown, the embodiment of the present invention discloses a longitudinal federated learning method based on virtual feature generation, and the steps of the method include:

[0058] Step S1. Initialize the task model based on vertical federated learning.

[0059] In an embodiment of the present invention, before step S1, it also includes: deploying N federated learning computing nodes, N is greater than 2, wherein 1 of the federated learning computing nodes is a server node, and N-1 of the federated learning computing nodes are client nodes.

[0060] Specifically, taking two-party vertical federated learning as an example, this invention case requires three servers to deploy federated learning computing nodes, one of which is used to deploy a server node (for the aggregation of intermediate parameters), and two are used to deploy two client nodes (for the training of local models). The two client nodes are fine-tuned and exchange the intermediate parameters of the two parties through the communication middleware, and the two client nodes correspond to one database to store relevant data. In fact, the present invention is applicable to any number of participants.

[0061] In vertical federated learning, data is divided into feature side and label side. Through vertical federated learning, joint learning is performed on different data sources, so that models can be built without leaking the original data.

[0062] In an embodiment of the present invention, a small amount of standard longitudinal federated learning can be performed to initialize the local task model, that is, before starting the local task model training, a small amount of data that can be used for federated learning is used for initial model training. The purpose is to provide an initial state for the local task model so that the subsequent federated learning process is more efficient and the model training effect is better.

[0063] Step S2. Train the task model on the client to obtain a first task model, wherein the task model is used to accept all input features x=[x_A, x_B] and output a predicted label y, wherein the input features include local real features x_A and external real features x_B.

[0064] Specifically, training the task model on the client to obtain a first task model includes:

[0065] Set the expected label, learn the conditional generation model on the client, input the local real feature and the expected label, and output the virtual external feature x`_B;

[0066] The task model is trained according to the local real feature x_A and the virtual external feature x`_B to obtain a first task model.

[0067] In an embodiment of the present invention, the generation model may be a multi-layer perceptron (MLP), which is composed of an input layer, a hidden layer, and an output layer. Through nonlinear transformation of multiple hidden layers, high-order feature representations of the data can be gradually extracted to generate virtual external features that the client does not have.

[0068] In an embodiment of the present invention, the generation model may also be VAE, GAN, DiffusionModel, etc. The generation model and task model may be replaced with any model according to the application scenario.

[0069] Preferably, the present invention introduces virtual external features, so that each participant has full feature data in federated learning, rather than all data, which ensures data privacy and security. In traditional vertical federated learning, participants share model parameters but not original data; horizontal federated learning focuses more on sharing data rather than model parameters. The present invention introduces virtual external features, so that each party can have complete feature data locally, and realizes data sharing among all parties in vertical federated learning.

[0070] In traditional vertical federated learning, the client can only access local features of the client and cannot share features of other clients. Therefore, it is necessary to linearly combine the features of each client and learn without exposing individual features. Traditional methods usually use model parameter aggregation to transmit model parameters between different clients. In order to reduce communication costs, the model complexity is limited and a single-layer neural network structure is used. Therefore, when the client trains the task model, an additional conditional generation model is learned to produce virtual external features. The introduced virtual external features can combine features of different dimensions through feature combination and crossover to obtain a nonlinear relationship with local real features, thereby improving the expressiveness of the model.

[0071] Specifically, training the task model on the client to obtain a first task model also includes:

[0072] Obtain a first prediction label according to the first task model;

[0073] Obtaining a task loss function based on the expected label and the first predicted label prediction loss;

[0074] The first task model and the conditional generation model are optimized using the task loss function as an objective function.

[0075] In an embodiment of the present invention, a negative log-likelihood loss function can be used to measure the difference between the expected label and the first predicted label. The loss function can be expressed as: L = -logp. The smaller the negative log-likelihood loss is, the better the model performance is.

[0076] Preferably, the task loss function can be used to optimize the conditional generation model and the local task model, and improve the performance of the local task model on the task with the help of the generated virtual external features x`_B.

[0077] Step S3: Aggregate the first task models on the server to obtain a global task model.

[0078] In an embodiment of the present invention, each client obtains a first task model after multiple local trainings, and sends the first task model to the server. After the server aggregates the first task model to obtain a global task model, the global task model is distributed to each client.

[0079] Completing aggregation on the server side can reduce communication between clients, reduce communication costs, and improve communication efficiency; the global task model can better handle the characteristics of different dimensions of each client and improve the generalization ability of the model.

[0080] Step S4: fine-tune the global task model according to the input features.

[0081] In an embodiment of the present invention, before the step of fine-tuning the global task model according to the input features, the method further includes:

[0082] After receiving the global task model, a round of communication is performed between the clients.

[0083] The purpose of communication is to use real features to fine-tune the global task model and improve the model's fitting ability.

[0084] In an embodiment of the present invention, fine-tuning the global task model according to the input features includes:

[0085] Inputting the input feature into the global task model to obtain a second prediction label;

[0086] The global task model is optimized according to the prediction loss of the true label and the second predicted label, wherein the true label is directly obtained from the data set.

[0087] Step S5. Repeat steps S2 to S4 to generate a final task model.

[0088] Repeat the above steps until the training is completed and the model achieves the desired performance.

[0089] The communication efficiency is calculated, where τ is the number of local training iterations of the model, T is the total number of iterations, α is the number of participants, batch_size is the size of each batch, if batch_size is 1, then each sample is iterated once, r is the number of rounds, and D is the sample size.

[0090] Taking the paradigm that requires two-way communication between participants as an example, the number of communications in each iteration of traditional vertical federated learning is α(α-1), and the total number of communications is Tα(α-1).

[0091] The solution of the present invention introduces the local model training phase, so there is no need for communication in the local τ iterations. After each τ iteration, the global model update phase is entered, and the vertical federated learning is converted to horizontal federated learning. Each client needs to communicate with the server twice, so the total number of communications in this phase is

[0092]

[0093] In order to improve the accuracy of the model, we then enter the fine-tuning phase, and the number of communications in this phase is

[0094]

[0095] Therefore, the total communication efficiency of the invention is

[0096]

[0097] The total number of iterations T can be calculated by the sample size, batch size and number of rounds, that is, It can be concluded that the communication efficiency of the final invention solution is

[0098]

[0099] The communication efficiency of federated learning using the present invention is reduced from Tα(α-1) to

[0100] like Figure 2 As shown, the embodiment of the present invention discloses a vertical federated learning system based on virtual feature generation, and its structure includes:

[0101] Initialization module, used to initialize the task model based on vertical federated learning;

[0102] A training module, used for training the task model on the client to obtain a first task model, wherein the task model is used for accepting all input features and outputting a predicted label, wherein the input features include local real features and external real features;

[0103] An aggregation module, used for aggregating the first task models on the server side to obtain a global task model;

[0104] A fine-tuning module, used for fine-tuning the global task model according to the input features;

[0105] The final generation module is used to repeat the steps to generate the final task model.

[0106] By using this system, the above-mentioned operation processing method can be executed and the corresponding technical effects can be achieved.

[0107] Embodiment 1

[0108] For the problem of credit card customer churn, the conventional approach is to establish an artificial neural network model to predict the possibility of customer churn and formulate recovery measures to prevent customer churn. However, in the traditional way, banks only have their own data, and the feature dimensions are not comprehensive, which may cause the model to be inaccurate; if data from other institutions is directly obtained, it will cause the risk of information leakage.

[0109] The method of the present invention can be used in the scenario of bank credit card customer churn prediction.

[0110] Step 1: Initialize the model;

[0111] Step 2. During the model training phase, the bank and the operator (other institutions such as e-commerce and government agencies can also be selected, and the operator is used as an example here) as two clients both maintain a task model locally, such as a 3-layer neural network model based on ANN. The input of this model is data including the bank's credit card categories, credit card spending limits, and operator customer calls, traffic, tariffs and other feature dimensions. In addition, both parties need to learn a generative model locally, such as GAN. Taking the bank side as an example, the input of the generative model is the bank's credit card categories and credit card spending limit data, and the output is virtual customer calls, traffic, tariffs and other limit data;

[0112] Take the bank side as an example: the bank uses real credit card data as input, obtains virtual operator data through the generation model, then merges the two data and passes them into the input layer, and uses the predicted labels obtained in the output layer to iteratively optimize the model. The local computer already has full-dimensional data, so there is no need to require the model to be linearly decomposable, and all types of models are supported;

[0113] Step 3: After multiple iterations locally, the model is sent to the server. The model has been iterated multiple times locally, which reduces the number of communications with the server and the other party and improves communication efficiency. The server aggregates the models of the two parties and sends the global model to the two parties.

[0114] Step 4: After receiving the global model, the two parties communicate interactively to fine-tune the model to ensure the accuracy of the model;

[0115] Step 5: Repeat steps 2 to 4 until the training is completed and the model achieves ideal performance.

[0116] Through the method described in the present invention, a bank credit card customer churn prediction model can be efficiently and accurately trained to help banks more accurately predict customer status, thereby better formulating and implementing the three measures.

[0117] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps in the method in the above embodiment, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the method in the above embodiment are implemented.

[0118] The embodiment of the present invention further provides an electronic device for executing the above method, as a device for implementing the method, and its structure is as follows: Figure 3As shown, the electronic device 300 includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one. It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the specific structure of the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. Optionally, the electronic device may also include a display screen 305 for displaying images, or receiving user operation instructions when necessary.

[0119] The processor 301 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 304 may include a receiver and a transmitter. The transceiver 304 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0120] Processor 301 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0121] The processor 301 may also include one or more processing units, for example, the processor 301 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 300. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 301 for storing instructions and data. In some embodiments, the memory in the processor 301 is a high-speed cache memory. The memory may store instructions or data that the processor 301 has just used or circulated. If the processor 301 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 301 is reduced, and the efficiency of the system is improved.

[0122] The processor 301 can run the method provided in the embodiment of the present application. The processor 301 can include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU can cooperate to execute the method provided in the embodiment of the present application, for example, part of the algorithm in the method is executed by the CPU, and another part of the algorithm is executed by the GPU to obtain faster processing efficiency.

[0123] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0124] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or may include a high-speed random access memory, or may include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0125] Optionally, the memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the method provided in any embodiment of the present application.

[0126] The memory 303 can be used to store computer executable program codes, which include instructions. The processor 301 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the memory 303. The memory 303 may include a program storage area and a data storage area. Among them, the program storage area may store the code of the operating system, application program, etc. The data storage area may store data created during the use of the electronic device 300 (such as images, videos, etc. collected by the camera application), etc.

[0127] The memory 303 may also store one or more computer programs corresponding to the method provided in the embodiment of the present application. The one or more computer programs are stored in the above-mentioned memory 303 and are configured to be executed by the one or more processors 301. The one or more computer programs include instructions, and the above-mentioned instructions can be used to execute each step in the above-mentioned corresponding embodiment.

[0128] Of course, the code of the method provided in the embodiment of the present application can also be stored in an external memory. In this case, the processor 301 can run the code of the method stored in the external memory through the external memory interface, and the processor 301 can control the running process.

[0129] The display screen 305 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 300 may include 1 or N display screens 305, where N is a positive integer greater than 1. The display screen 305 can be used to display information input by a user or information provided to a user and various graphical user interfaces (GUI). For example, the display screen 305 can display photos, videos, web pages, or files.

[0130] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0131] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0133] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0135] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A longitudinal federated learning method based on virtual feature generation, characterized in that: include: S1. Initialize the task model based on vertical federated learning; S2. Train the task model on the client to obtain a first task model, wherein the task model is used to accept all input features and output a predicted label, wherein the input features include local real features and external real features; S3. Aggregate the first task model on the server to obtain a global task model; S4. fine-tuning the global task model according to the input features; S5. Repeat steps S2 to S4 to generate the final task model.

2. The method according to claim 1, characterized in that Prior to the method, it also includes: deploying N federated learning computing nodes, N is greater than 2, wherein 1 of the federated learning computing nodes is a server node and N-1 of the federated learning computing nodes are client nodes.

3. The method according to claim 1, characterized in that The step of training the task model on the client to obtain a first task model includes: Setting the expected label, learning the conditional generative model on the client, inputting the local real feature and the expected label, and outputting the virtual external feature; The task model is trained according to the local real features and the virtual external features to obtain a first task model.

4. The method according to claim 3, characterized in that The method further comprises: Obtain a first prediction label according to the first task model; Obtaining a task loss function based on the expected label and the first predicted label prediction loss; The first task model and the conditional generation model are optimized using the task loss function as an objective function.

5. The method according to claim 1, characterized in that After the server aggregates the first task model to obtain a global task model, the method further includes: The global task model is distributed to clients. Before fine-tuning the global task model according to the input features, the method further includes: After receiving the global task model, the clients communicate with each other.

6. The method according to claim 1, characterized in that The fine-tuning of the global task model according to the input features includes: Inputting the input feature into the global task model to obtain a second prediction label; The global task model is optimized according to the prediction loss of the true label and the second predicted label, wherein the true label is directly obtained from the data set.

7. A vertical federated learning system based on virtual feature generation, characterized in that: include: Initialization module, used to initialize the task model based on vertical federated learning; A training module, used for training the task model on the client to obtain a first task model, wherein the task model is used for accepting all input features and outputting a predicted label, wherein the input features include local real features and external real features; An aggregation module, used for aggregating the first task models on the server side to obtain a global task model; A fine-tuning module, used for fine-tuning the global task model according to the input features; The final generation module is used to repeat the steps to generate the final task model.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store program instructions; The processor is configured to execute the method according to any one of claims 1 to 6 by calling program instructions.

10. A computer program product comprising a computer program and / or instructions, characterized in that: When the computer program and / or the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Cited By

  • Longitudinal federal learning training method and system, application, terminal and medium

    CN121235033A