Method and terminal device for vertical federated learning linear regression

Abstract

The present invention provides a longitudinal federated learning linear regression method and a terminal device, the method comprising: selecting a first data node among multiple data nodes as an initiator and selecting data nodes other than the first data node as responders; obtaining the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, where i is a natural number greater than 0; based on the i-th moment parameter of the responder, obtaining the i-th moment response value through a first function calculation, and based on the i+1-th moment parameter of the responder, obtaining the i+1-th moment response value through a first function calculation; based on the i-th moment parameter and the i-th moment response value of the initiator, obtaining the i-th moment prediction value through a second function calculation, and based on the i+1-th moment parameter and the i+1-th moment response value of the initiator, obtaining the i+1-th moment prediction value through a second function calculation; if the difference between the i+1-th moment prediction value and the i-th moment prediction value is not greater than a preset value, sending a termination signal and stopping training.

Description

Technical Field

[0001] The present invention belongs to the technical field of linear regression, and in particular to a method and terminal device for vertical federated learning linear regression. Background Art

[0002] PySyft is an algorithm framework of the federated learning open source community that provides communication between data nodes and the PyTorch algorithm library, but it has not yet implemented linear regression.

[0003] The existing technical solution, linear regression implemented by another framework FATE, mainly relies on direct communication between data nodes, rather than relying on trusted server data nodes. This solution requires frequent use of homomorphic encryption to ensure privacy protection, which may lead to encryption computing efficiency consumption. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and a terminal device for vertical federated learning linear regression to solve the problem of low efficiency in the prior art.

[0005] A first aspect of an embodiment of the present invention provides a method for vertical federated learning linear regression, which includes:

[0006] S101: Select a first data node among multiple data nodes as an initiator and select data nodes other than the first data node as responders;

[0007] S102: Obtaining the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, where i is a natural number greater than 0;

[0008] S103: Based on the i-th moment parameter of the responder, calculate the i-th moment response value through the first function; based on the i+1-th moment parameter of the responder, calculate the i+1-th moment response value through the first function;

[0009] S104: Based on the i-th moment parameter of the initiator and the i-th moment response value, obtain the i-th moment prediction value by using a second function calculation; based on the i+1-th moment parameter of the initiator and the i+1-th moment response value, obtain the i+1-th moment prediction value by using the second function calculation;

[0010] S105: If the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than a preset value, a termination signal is issued and training is stopped.

[0011] As described above, in the method of longitudinal federated learning linear regression, the i-th moment parameter and the i+1-th moment parameter both include the weight and data volume of each data node among the multiple data nodes.

[0012] The method of longitudinal federated learning linear regression as described above, wherein the first function is the sum of responses of the product of the respondent's weight and the amount of data.

[0013] The method of longitudinal federated learning linear regression as described above is characterized in that the second function is the sum of the product of the initiator's weight and the amount of data and the sum of the responses minus the target value.

[0014] The method for vertical federated learning linear regression as described above is characterized in that the method further comprises:

[0015] Distribute the prediction value at the i-th moment and the prediction value at the i+1-th moment to each data node;

[0016] Based on the predicted value at the i-th moment and the amount of data in the parameter at the i-th moment of each data node, obtaining the gradient value at the i-th moment of each data node through a third function;

[0017] Based on the i+1th moment prediction value and the amount of data in the i+1th moment parameter of each data node, obtaining the i+1th moment gradient value of each data node through the third function;

[0018] If the difference between the gradient value of each data node at the i+1th moment and the gradient value of the corresponding data node at the ith moment is not greater than the preset gradient value, a termination signal is sent to stop the training.

[0019] In the method of longitudinal federated learning linear regression as described above, if the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is greater than a preset value, steps S102 to S104 are repeated until the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is no greater than a preset value.

[0020] A second aspect of an embodiment of the present invention provides a device for vertical federated learning linear regression, comprising:

[0021] A selection module, configured to select a first data node among a plurality of data nodes as an initiator and select data nodes other than the first data node as responders;

[0022] an acquisition module, configured to acquire the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, wherein i is a natural number greater than 0;

[0023] A first calculation module, configured to calculate a response value at the i-th moment by using a first function based on the i-th moment parameter of the responder, and to calculate a response value at the i+1th moment by using the first function based on the i+1-th moment parameter of the responder;

[0024] A second calculation module, which is used to calculate the predicted value of the i-th moment through a second function based on the i-th moment parameter of the initiator and the i-th moment response value, and to calculate the predicted value of the i+1-th moment through the second function based on the i+1-th moment parameter of the initiator and the i+1-th moment response value;

[0025] A determination module is used to send a termination signal and stop training in response to the difference between the predicted value at the i+1th moment and the predicted value at the ith moment being not greater than a preset value.

[0026] The device for longitudinal federated learning linear regression as described above, wherein the determination module is also used for: in response to the difference between the prediction value at the i+1th moment and the prediction value at the i-th moment being greater than a preset value, the acquisition module, the first calculation module and the second calculation module repeatedly operate until the difference between the prediction value at the i+1th moment and the prediction value at the i-th moment is not greater than a preset value.

[0027] A third aspect of an embodiment of the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0028] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program implements the steps of the above method when executed by a processor.

[0029] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: reducing encryption calculations during the training process and improving execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 is a flow chart of a method for longitudinal federated learning linear regression provided by an embodiment of the present invention;

[0032] Figure 2 It is a schematic diagram of the implementation flow of the method for longitudinal federated learning linear regression provided by an embodiment of the present invention;

[0033] Figure 3is a schematic diagram of a device for longitudinal federated learning linear regression provided by an embodiment of the present invention;

[0034] Figure 4 It is a schematic diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0036] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a method for longitudinal federated learning linear regression, which includes:

[0038] S101: Select a first data node among a plurality of data nodes as an initiator and select data nodes other than the first data node as responders.

[0039] S102: Obtain the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, where i is a natural number greater than 0.

[0040] Specifically, the parameters at the i-th moment and the parameters at the i+1-th moment both include the weight and data volume of each data node among the multiple data nodes, wherein the types contained in the parameters corresponding to any moment are the same, both including the weight and data volume of each data node among the multiple data nodes.

[0041] S103: Based on the i-th moment parameter of the responder, obtain the i-th moment response value through the first function calculation; based on the i+1-th moment parameter of the responder, obtain the i+1-th moment response value through the first function calculation.

[0042] Specifically, the first function is the sum of responses of the product of the respondent's weight and the amount of data.

[0043] S104: Based on the i-th moment parameters of the initiator and the i-th moment response value, the i-th moment prediction value is calculated by a second function; based on the i+1-th moment parameters of the initiator and the i+1-th moment response value, the i+1-th moment prediction value is calculated by the second function.

[0044] Specifically, the second function is the product of the initiator's weight and data volume and the sum of the responses minus the target value, where the target value is a specific target value set by technical personnel in this field according to the application environment. The target value is different in different application environments and is not limited here.

[0045] S105: If the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than a preset value, a termination signal is issued and training is stopped.

[0046] Specifically, if the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than the preset value, a termination signal is issued and training is stopped; if the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is greater than the preset value, steps S102 to S104 are repeated until the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than the preset value.

[0047] Furthermore, the method for longitudinal federated learning linear regression of the present invention further includes between step S104 and step S105:

[0048] Distribute the predicted value at the i-th moment and the predicted value at the i+1-th moment to each data node;

[0049] Based on the prediction value at the i-th moment and the amount of data in the parameters at the i-th moment of each data node, the gradient value at the i-th moment of each data node is obtained by a third function;

[0050] Based on the predicted value at the i+1th moment and the amount of data in the parameters at the i+1th moment of each data node, the gradient value at the i+1th moment of each data node is obtained by a third function;

[0051] If the difference between the gradient value of each data node at the i+1th moment and the gradient value of the corresponding data node at the ith moment is not greater than the preset gradient value, a termination signal is sent to stop the training.

[0052] Specifically, if the difference between the gradient value of each data node at the i+1th moment and the gradient value of the corresponding data node at the i-th moment is not greater than the preset gradient value, a termination signal is issued and the training is stopped; if the difference between the gradient value of each data node at the i+1th moment and the gradient value of the corresponding data node at the i-th moment is greater than the preset gradient value, steps S102 to S104 are repeated until the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than the preset value.

[0053] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0054] Now combined Figure 2A specific embodiment of the method for longitudinal federated learning linear regression of the present invention is described in detail. This description is for clarifying the present invention and is not intended to limit the present invention.

[0055] like Figure 2 As shown, the method of longitudinal federated learning linear regression of the present invention comprises the following steps:

[0056] 1) Among multiple data nodes (for example, A, B, and C), select A as the initiator, and B and C as responders. At the same time, the three client data nodes A, B, and C communicate with the server (Server) in a unified manner. Of course, any of the B and C data nodes can also be selected as the initiator. If B is the initiator, A and C are selected as responders. There is no restriction on which data node is the initiator. Of course, two data nodes can also be selected as initiators, and there is no specific restriction here.

[0057] 2) The three client data nodes each calculate W·X based on their own weight Wa / Wb / Wc and their own sample data volume Xa / Xb / Xc. Responders B and C send the calculation results to the Server. The Server calculates the sum using the following formula, that is, sum = Wb·Xb + Wb·Xc, and forwards the sum to the initiator A. Figure 2 Step 1.

[0058] 3) Data node A receives sum and calculates the prediction error d by the following formula, i.e., d=Wb·Xb+sum-y, where y in the process of calculating d refers to the target value, which is a specific target value set by technicians in this field according to the application environment. In different application environments, the target value is different, and the target value is not limited here. d is forwarded to data nodes B and C through the Server. After obtaining d, A, B, and C calculate the gradients Ga, Gb, and Gc corresponding to their respective weights, as shown in Step 2 in the figure.

[0059] 4) As shown in Step 3, all client data nodes upload their gradients to the server. The server determines whether the global model has converged based on the previous and current gradients. If so, it sends a termination signal to stop training. Otherwise, continue with step 2) above and iterate training.

[0060] Specifically, in step 4), the last gradient situation refers to the result of calculation of the parameters corresponding to each data node at the last moment using step 2) and step 3), and when determining whether convergence has occurred, if the difference between the current gradient situation and the previous gradient situation is not greater than a preset value, convergence is determined, and the preset value may be 0.01, 0.001, or other values. In different application environments, the preset value may be different or the same, and the preset value is not limited herein.

[0061] like Figure 3 As shown, the present invention also provides a device 3 for longitudinal federated learning linear regression, which includes:

[0062] A selection module 301, which is used to select a first data node among multiple data nodes as an initiator and select data nodes other than the first data node as responders;

[0063] An acquisition module 302 is used to acquire the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, where i is a natural number greater than 0;

[0064] A first calculation module 303, which is used to calculate the response value at the i-th moment through the first function based on the i-th moment parameter of the responder, and to calculate the response value at the i+1th moment through the first function based on the i+1-th moment parameter of the responder;

[0065] A second calculation module 304 is configured to calculate the predicted value of the i-th moment by using a second function based on the i-th moment parameter and the i-th moment response value of the initiator, and to calculate the predicted value of the i+1-th moment by using a second function based on the i+1-th moment parameter and the i+1-th moment response value of the initiator;

[0066] The determination module 305 is used to send a termination signal and stop training in response to the difference between the predicted value at the i+1th moment and the predicted value at the ith moment being not greater than a preset value.

[0067] The various modules of the device for longitudinal federated learning linear regression in this embodiment correspond to the various steps of the above method, which will not be repeated here.

[0068] Figure 4 Schematic diagram of a terminal device provided by an embodiment of the present invention. Figure 4 As shown, the terminal device 4 of this embodiment includes: a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40, such as a longitudinal federated learning linear regression program. When the processor 40 executes the computer program 42, the steps in the above-mentioned longitudinal federated learning linear regression method embodiments are implemented, such as Figure 1Alternatively, when the processor 40 executes the computer program 42, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 3 Functions of modules 301 to 305 are shown.

[0069] Exemplarily, the computer program 42 may be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 42 in the terminal device 4.

[0070] The terminal device 4 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4 It is only an example of the terminal device 4 and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0071] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0072] The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used to store the computer program and other programs and data required by the terminal device. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0073] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0074] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0076] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0079] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0080] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for longitudinal federated learning linear regression, characterized in that: include: S101: Select a first data node among multiple data nodes as an initiator and select data nodes other than the first data node as responders; S102: Obtaining the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, where i is a natural number greater than 0; S103: Based on the i-th moment parameter of the responder, calculate the i-th moment response value through a first function, and based on the i+1-th moment parameter of the responder, calculate the i+1-th moment response value through the first function; the first function is the total response of the product of the responder's weight and the data volume; S104: Based on the i-th moment parameter of the initiator and the i-th moment response value, a predicted value of the i-th moment is calculated by a second function; based on the i+1-th moment parameter of the initiator and the i+1-th moment response value, a predicted value of the i+1-th moment is calculated by the second function; the second function is the product of the weight and data volume of the initiator and the sum of the responses minus the target value; S105: If the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than a preset value, a termination signal is issued and training is stopped.

2. The method for vertical federated learning linear regression according to claim 1, characterized in that: The i-th moment parameter and the i+1-th moment parameter both include the weight and data volume of each data node among the multiple data nodes.

3. The method for longitudinal federated learning linear regression according to claim 2, characterized in that: The method further comprises: Distribute the prediction value at the i-th moment and the prediction value at the i+1-th moment to each data node; Based on the predicted value at the i-th moment and the amount of data in the parameter at the i-th moment of each data node, obtaining the gradient value at the i-th moment of each data node through a third function; Based on the i+1th moment prediction value and the amount of data in the i+1th moment parameter of each data node, obtaining the i+1th moment gradient value of each data node through the third function; If the difference between the gradient value of each data node at the i+1th moment and the gradient value of the corresponding data node at the ith moment is not greater than the preset gradient value, a termination signal is sent to stop the training.

4. The method for longitudinal federated learning linear regression according to claim 1, characterized in that: If the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is greater than the preset value, steps S102 to S104 are repeated until the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is not greater than the preset value.

5. A device for longitudinal federated learning linear regression, characterized in that: include: A selection module, configured to select a first data node among a plurality of data nodes as an initiator and select data nodes other than the first data node as responders; an acquisition module, configured to acquire the i-th moment parameter corresponding to the i-th moment and the i+1-th moment parameter corresponding to the i+1-th moment of the initiator and the responder, wherein i is a natural number greater than 0; A first calculation module, which is used to calculate the i-th moment response value through a first function based on the i-th moment parameter of the responder, and to calculate the i+1-th moment response value through the first function based on the i+1-th moment parameter of the responder; the first function is the response sum of the product of the weight of the responder and the data volume; A second calculation module, which is used to calculate the predicted value of the i-th moment through a second function based on the i-th moment parameter of the initiator and the i-th moment response value, and calculate the predicted value of the i+1-th moment through the second function based on the i+1-th moment parameter of the initiator and the i+1-th moment response value; the second function is the sum of the product of the weight and data volume of the initiator and the sum of the responses minus the target value; A determination module is used to send a termination signal and stop training in response to the difference between the predicted value at the i+1th moment and the predicted value at the ith moment being not greater than a preset value.

6. The device for vertical federated learning linear regression according to claim 5, characterized in that: The determination module is also used for: in response to the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment being greater than a preset value, the acquisition module, the first calculation module and the second calculation module repeatedly operate until the difference between the predicted value at the i+1th moment and the predicted value at the i-th moment is no greater than a preset value.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images