Transverse federal model construction optimization method, system, device, medium and product

Abstract

This application discloses a method, system, device, medium, and product for constructing and optimizing a horizontal federated model, applied to participating devices. The method includes: acquiring local samples; inputting the local samples into a private privacy protection module and performing periodic sample transformation on the local samples to obtain periodically transformed samples; inputting the periodically transformed samples into a local shared neural network and performing local iterative training optimization on the private privacy protection module and the local shared neural network; acquiring the local network parameters of the local shared neural network and uploading the local network parameters to a federated server to perform iterative optimization on the private privacy protection module and the local shared neural network based on horizontal federated learning. This application solves the technical problem of how to improve the communication efficiency between participating devices in the horizontal federated modeling process while protecting data privacy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and in particular to a method, system, device, medium and product for optimizing the construction of a horizontal federated model. Background Technology

[0002] With the continuous development of fintech, especially internet fintech, more and more technologies (such as distributed systems and artificial intelligence) are being applied in the financial field. However, the financial industry is also placing higher demands on technology, such as on the distribution of tasks to be completed.

[0003] Federated learning, as a distributed machine learning approach, can solve the "data silo" problem while protecting the data privacy of participating devices. Currently, homomorphic encryption is commonly used for privacy protection in the process of building models based on horizontal federated learning. However, both homomorphic encryption and secret sharing will lead to a surge in the amount of communication data between participating devices, affecting the communication efficiency between them. Summary of the Invention

[0004] The main purpose of this application is to provide a method, system, device, medium and product for optimizing the construction of horizontal federation models, which aims to solve the technical problem of how to improve the communication efficiency between the devices of the participating parties in the horizontal federation modeling process while protecting data privacy.

[0005] To achieve the above objectives, this application provides a horizontal federation model construction optimization method, applied to participating party devices, the horizontal federation model construction optimization method comprising:

[0006] Local samples are obtained by inputting them into a private privacy protection module, which performs periodic sample transformation on the local samples to obtain periodically transformed samples.

[0007] By inputting the periodic transformation samples into a local shared neural network, the private privacy protection module and the local shared neural network are subjected to local iterative training and optimization.

[0008] The local network parameters of the local shared neural network are obtained, and the local network parameters are uploaded to the federation server. The federation server is used to aggregate the local network parameters uploaded by each of the participating devices into federation network parameters.

[0009] Receive the federated network parameters sent by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters;

[0010] Return to execution steps: Obtain local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, until the horizontal federated learning modeling is detected to be complete.

[0011] Optionally, the private privacy protection module includes a private periodic neural network and a private noise module.

[0012] The step of inputting the local sample into the private privacy protection module and performing periodic sample transformation on the local sample to obtain periodically transformed samples includes:

[0013] The local samples are input into the private periodic neural network, and periodic sample mapping is performed on the local samples to obtain periodic mapped samples.

[0014] Based on the proprietary noise module, noise is added to the periodic mapping sample to obtain the periodic transformation sample.

[0015] Optionally, the private periodic neural network includes neural network parameters and a periodic activation function.

[0016] The step of inputting the local samples into the private periodic neural network and performing periodic-based sample mapping on the local samples to obtain periodic mapped samples includes:

[0017] Based on the neural network parameters, the local samples are mapped to obtain mapped samples;

[0018] The periodic activation function is used to periodically activate the mapped sample to obtain the periodic mapped sample.

[0019] Optionally, the horizontal federation model construction optimization method further includes:

[0020] Obtain the sample to be predicted, and perform a periodic sample transformation on the sample to be predicted according to the private privacy protection module to obtain the first periodic transformation prediction sample.

[0021] Based on the local shared neural network, sample prediction is performed on the first periodic transformation prediction sample to obtain the first sample prediction result.

[0022] Optionally, the horizontal federation model construction optimization method further includes:

[0023] The client receives a second periodic transformation prediction sample and performs sample prediction on the second periodic transformation prediction sample according to the local shared neural network to obtain a second sample prediction result. The second periodic transformation prediction sample is obtained by the client performing periodic sample transformation on the sample to be predicted based on the target privacy protection module determined by the private privacy protection module issued by the participating device.

[0024] The prediction result of the second sample is fed back to the client.

[0025] Optionally, prior to the step of receiving the second periodic transformation prediction sample sent by the client, the lateral federated model construction optimization method further includes:

[0026] The private privacy protection module and the preset noise threshold are sent to the client so that the client can adjust the private noise module in the private privacy protection module according to the local additional noise determined by the preset noise threshold, thereby obtaining the target privacy protection module.

[0027] This application also provides a horizontal federated learning system, the horizontal federated learning system comprising:

[0028] At least one participating device is used to acquire local samples, input the local samples into a private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, input the periodically transformed samples into a local shared neural network, and perform local iterative training and optimization on the private privacy protection module and the local shared neural network; acquire the local network parameters of the local shared neural network, and upload the local network parameters to the federated server; receive the federated network parameters issued by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters; return to the execution steps: acquire local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, until the lateral federated learning modeling is detected to be complete;

[0029] A federated server is used to aggregate the local network parameters uploaded by each of the participating devices into federated network parameters.

[0030] Optionally, the horizontal federated learning system also includes a client.

[0031] The participating device is also used to distribute the private privacy protection module to the client; receive the periodic transformation prediction sample sent by the client; perform sample prediction on the periodic transformation prediction sample according to the local shared neural network to obtain the sample prediction result; and feed back the sample prediction result to the client.

[0032] The client is used to perform periodic sample transformation on the sample to be predicted based on the target privacy protection module determined by the private privacy protection module, to obtain periodic transformed prediction samples; send the periodic transformed prediction samples to the participating device; and receive the sample prediction results fed back by the participating device.

[0033] Optionally, the private privacy protection module includes a private periodic neural network and a private noise module, and the participating device is further used for:

[0034] The local samples are input into the private periodic neural network, and periodic sample mapping is performed on the local samples to obtain periodic mapped samples.

[0035] Based on the proprietary noise module, noise is added to the periodic mapping sample to obtain the periodic transformation sample.

[0036] Optionally, the private periodic neural network includes neural network parameters and a periodic activation function, and the participating device is further configured to:

[0037] Based on the neural network parameters, the local samples are mapped to obtain mapped samples;

[0038] The periodic activation function is used to periodically activate the mapped sample to obtain the periodic mapped sample.

[0039] Optionally, the participating device is further used for:

[0040] Obtain the sample to be predicted, and perform a periodic sample transformation on the sample to be predicted according to the private privacy protection module to obtain the first periodic transformation prediction sample.

[0041] Based on the local shared neural network, sample prediction is performed on the first periodic transformation prediction sample to obtain the first sample prediction result.

[0042] Optionally, the participating device is further used for:

[0043] The private privacy protection module and the preset noise threshold are sent to the client.

[0044] Optionally, the client is also used for:

[0045] Based on the local additional noise determined by the preset noise threshold, the private noise module in the private privacy protection module is adjusted to obtain the target privacy protection module.

[0046] This application also provides an electronic device, the electronic device comprising: a memory, a processor, and a program of the horizontal federation model construction optimization method stored in the memory and executable on the processor, wherein when the program of the horizontal federation model construction optimization method is executed by the processor, it can implement the steps of the horizontal federation model construction optimization method as described above.

[0047] This application also provides a computer-readable storage medium storing a program implementing a horizontal federation model construction optimization method, wherein when the program is executed by a processor, it implements the steps of the horizontal federation model construction optimization method as described above.

[0048] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the horizontal federation model construction optimization method described above.

[0049] This application provides a method, system, device, medium, and product for constructing and optimizing a horizontal federated model. In this horizontal federated learning process, each participating device is equipped with a local shared neural network and a private privacy protection module. The participating device can input the local samples into the privacy protection module to perform periodic sample transformations on the local samples, obtaining periodically transformed samples. The periodically transformed samples are then input into the local shared neural network to perform local iterative training and optimization on the privacy protection module and the local shared neural network. The local network parameters of the local shared neural network are obtained and uploaded to a federated server. The federated server aggregates the local network parameters uploaded by each participating device into federated network parameters. The application receives the federated network parameters from the federated server and updates the local network parameters of the local shared neural network to the federated network parameters. The application then returns to the execution steps: obtaining local samples, inputting the local samples into the privacy protection module, performing periodic sample transformations on the local samples, obtaining periodically transformed samples, until the horizontal federated learning modeling is detected as complete. Since the periodic transformation samples are obtained through periodic sample transformation, one periodic transformation sample corresponds to the input samples of multiple private privacy protection modules. Therefore, even if the external system reverse-engineers the periodic transformation samples based on the model parameters of the local shared neural network, it is difficult to deduce which period's private privacy protection module's input sample the periodic transformation sample corresponds to. The difficulty of reverse-engineering is equivalent to the difficulty of fault-tolerant learning. Thus, data privacy protection for participating devices is achieved during the horizontal federated learning modeling process. Furthermore, the local network parameters uploaded by participating devices to the federated server are all plaintext data. Compared to the existing technology that uses homomorphic encryption for data privacy protection during the horizontal federated learning modeling process, the amount of plaintext data in this application is much smaller than that of homomorphically encrypted ciphertext data. Therefore, the amount of communication data transmitted between participating devices and the federated server during the horizontal federated learning modeling process can be reduced, improving the communication efficiency during the horizontal federated learning process. This solves the technical problem of how to improve the communication efficiency between participating devices during the horizontal federated learning modeling process while ensuring data privacy protection. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating the first embodiment of the optimization method for constructing a horizontal federation model in this application.

[0053] Figure 2 This is a schematic diagram of the process of periodic sample transformation based on a private privacy protection module in the optimization method for constructing the horizontal federation model in this application;

[0054] Figure 3 A schematic diagram illustrating the process of constructing a private privacy-preserving module and a local shared neural network based on horizontal federated learning in the optimization method for constructing the horizontal federated model in this application;

[0055] Figure 4 This is a schematic diagram of the system structure of an embodiment of the horizontal federated learning system of this application;

[0056] Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the horizontal federation model construction and optimization method in this application embodiment.

[0057] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0058] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] Example 1

[0060] This application provides a method for optimizing the construction of a horizontal federation model. In the first embodiment of this method, refer to... Figure 1 The method in this embodiment is applied to participating party devices, and the horizontal federation model construction optimization method includes:

[0061] Step S10: Obtain local samples. By inputting the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples.

[0062] Step S20: By inputting the periodic transformation samples into the local shared neural network, the private privacy protection module and the local shared neural network are subjected to local iterative training optimization.

[0063] Step S30: Obtain the local network parameters of the local shared neural network and upload the local network parameters to the federation server, wherein the federation server is used to aggregate the local network parameters uploaded by each participating device into federation network parameters;

[0064] Step S40: Receive the federated network parameters sent by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters;

[0065] Step S50, return to the execution steps: obtain local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodic transformed samples, until the horizontal federated learning modeling is detected to be completed.

[0066] In this embodiment, it should be noted that the participating device is a participant in horizontal federated learning, the federated server is a coordinator in horizontal federated learning, and the private privacy protection module is used to perform periodic sample transformation on local samples, converting each sample feature value in the local samples into a periodic transformation feature value that conforms to periodic changes, thereby obtaining periodic transformed samples. Since the periodic transformation feature value conforms to periodic changes, the sample feature value corresponding to one periodic transformation feature value is not unique, that is, one periodic transformation feature value corresponds to multiple different sample feature values. Therefore, even if the server knows the periodic transformed samples and the module parameters of the private privacy protection module, it is difficult to deduce the unique sample feature value corresponding to each periodic transformation feature value in the periodic transformed samples, and thus it is difficult to deduce the unique local sample corresponding to the periodic transformed samples. The difficulty of deduction is equivalent to the difficulty of fault-tolerant learning, so the data privacy of local samples in the participating device can be well protected. The participating devices are also equipped with a local shared neural network. This local shared neural network is synchronously trained and updated with the private privacy protection module. The local network parameters of the local shared neural network are directly uploaded to the federated server in plaintext data for aggregation, so as to enable all participating devices to share the local neural network parameters.

[0067] As an example, steps S10 to S50 include: obtaining local samples and corresponding sample labels, wherein the local sample consists of at least one sample feature value; inputting the local sample into a private privacy protection module to perform a periodic sample transformation on the local sample, so as to convert each sample feature value in the local sample into a feature value that conforms to periodic changes, thereby obtaining a periodically transformed sample corresponding to the local sample; inputting the periodically transformed sample into a local shared neural network for sample prediction, thereby obtaining a sample prediction result; calculating the corresponding model loss based on the difference between the sample prediction result and the sample label; if the model loss converges, it is determined that the lateral federated learning modeling is completed; if the model loss does not converge, the model gradient calculated based on the model loss is used to adjust the private privacy protection module and the local shared neural network. The network performs backpropagation updates; the execution steps are returned: obtain local samples and their corresponding sample labels, and check whether the iteration update count of the private privacy protection module and the local shared neural network has reached the preset iteration update count. If it has, obtain the local network parameters of the local shared neural network and upload the local network parameters to the federated server. The federated server is used to aggregate the local network parameters uploaded by each participating device into federated network parameters. The aggregation method can be weighted summation or weighted average, etc. The federated network parameters are received from the federated server, and the local network parameters of the local shared neural network are replaced and updated with the federated network parameters. The execution steps are returned: obtain local samples and their corresponding sample labels for the next round of iteration training until the horizontal federated learning modeling is detected to be completed.

[0068] As an example, in order for the privacy protection module to learn the knowledge at the devices of each participating party in lateral federated learning more efficiently, after the step of replacing and updating the local network parameters of the local shared neural network with the federated network parameters, the method further includes:

[0069] While keeping the network parameters of the local shared neural network unchanged, the privacy protection module is trained and updated based on the current local samples until the privacy protection module meets the preset training update termination condition. Then, the process returns to the next step: obtaining local samples and their corresponding sample labels for the next round of iterative training, until the lateral federated learning modeling is detected as complete. The preset training update termination condition can be reaching a preset number of training updates or model loss convergence. The current local samples are those obtained during this iteration.

[0070] It should be noted that currently, when building models based on horizontal federated learning, in order to protect the data privacy of participating devices, horizontal federated learning encryption is usually based on homomorphic encryption or secret sharing. However, the data volume of the encrypted ciphertext data is much larger than that of the plaintext data, which greatly increases the amount of data transmitted between each participating device and the federated server, and also greatly increases the amount of computational data for each participating device and the federated server. This will greatly affect the communication efficiency between each participating device and the federated server, and also greatly affect the data computation efficiency of each participating device and the federated server in the horizontal federated learning modeling process. In this embodiment, the local network parameters transmitted by the participating devices to the federated server are plaintext data, not ciphertext data. This effectively reduces the amount of communication data between the participating devices and the federated server, as well as the amount of computational data required by each device and server during the horizontal federated learning modeling process. Therefore, it improves the communication efficiency between the participating devices and the federated server, and the data computation efficiency of each device and server during the horizontal federated learning modeling process. Furthermore, although the federated server directly obtains the local network parameters of the shared neural network from each participating device (which are plaintext data), and can deduce the input data of the shared neural network (i.e., periodically changing samples), it does not know the module parameters in the private privacy protection module. The server cannot deduce the local samples of the participating devices; and even if the federated server obtains the module parameters in the private privacy protection module, since the periodic transformation samples are subject to periodic changes, the sample feature value corresponding to each feature value in the same periodic transformation sample is not unique. The federated server has difficulty deduce a unique local sample based on the periodic transformation samples, and the difficulty of deduction is equivalent to the difficulty of fault-tolerant learning. Therefore, the data privacy of the participating devices can be well protected. In summary, the embodiment of this application constructs a private privacy protection module and a local shared neural network based on horizontal federated learning, which not only protects the data privacy of the participating devices, but also improves the communication efficiency between the participating devices and the federated server, as well as improves the data computation efficiency of the participating devices and the federated server in the horizontal federated learning modeling process.

[0071] The private privacy protection module includes a private periodic neural network and a private noise module.

[0072] The step of inputting the local sample into the private privacy protection module and performing periodic sample transformation on the local sample to obtain periodically transformed samples includes:

[0073] Step S11: Input the local sample into the private periodic neural network, perform periodic sample mapping on the local sample, and obtain periodic mapped samples;

[0074] Step S12: Based on the private noise module, noise is added to the periodic mapping sample to obtain the periodic transformation sample.

[0075] In this embodiment, it should be noted that the private privacy protection module may consist of a private periodic neural network and a private noise module. The private periodic neural network is used to perform periodic sample transformation on the local samples, and the private noise module is used to add noise to the output of the private periodic neural network.

[0076] As an example, steps S11 to S12 include: inputting the local samples into the private periodic neural network, performing periodic sample mapping on the local samples to convert the feature values ​​of each sample in the local samples into periodic transformation feature values ​​that conform to periodic changes, thereby obtaining the periodic mapping sample corresponding to the local samples; and inputting the periodic mapping sample into the private noise module to add corresponding additional noise to the periodic mapping sample to obtain the periodic transformation sample. In this embodiment, a noise module is set in the private privacy protection module. Thus, in addition to converting each sample feature value in the local samples into periodic transformation feature values ​​that conform to periodic changes, additional noise can be added. Since the private noise module is privately held by the participating device, if the federated server cannot obtain this additional noise, it can further increase the difficulty for the federated server to deduce the local samples of the participating device, thereby improving the data privacy protection effect during the horizontal federated learning modeling process in this embodiment.

[0077] The private periodic neural network includes neural network parameters and a periodic activation function.

[0078] The step of inputting the local samples into the private periodic neural network and performing periodic-based sample mapping on the local samples to obtain periodic mapped samples includes:

[0079] Step S111: Based on the neural network parameters, perform sample mapping on the local samples to obtain mapped samples;

[0080] Step S112: Periodically activate the mapped sample according to the periodic activation function to obtain the periodic mapped sample.

[0081] In this embodiment, it should be noted that the private periodic neural network can be composed of neural network parameters and a periodic activation function. The neural network parameters are used to perform linear transformation on the local samples, and the periodic activation function is a periodic function used to activate the local samples after linear transformation, thereby outputting periodic mapping samples.

[0082] As an example, steps S111 to S112 include: performing a linear transformation on the feature values ​​of each sample in the local sample according to the neural network parameters to obtain a linearly transformed sample; and activating each feature value in the linearly transformed sample according to the periodic activation function to obtain a periodically mapped sample. Since each feature value in the linearly transformed sample is activated by inputting a periodic activation function, each feature value in the periodically mapped sample exhibits periodic changes. Each feature value in the periodically mapped sample does not correspond one-to-one with each feature value in the local sample. For example, assuming the periodic activation function is sinX, if a feature value in the periodically mapped sample is 1, then the corresponding value of X can be multiple. Therefore, even if the neural network parameters and the periodically mapped sample are obtained externally, it is difficult to deduce the feature values ​​of the sample in the local sample. Moreover, in this embodiment, additional noise is applied to the periodically mapped sample, further increasing the difficulty for external parties to deduce the local sample.

[0083] Furthermore, it should be noted that current data encryption methods such as homomorphic encryption or secret sharing are non-linear data transformation processes. Therefore, the data volume of the final encrypted ciphertext is usually much larger than that of the plaintext. However, in this embodiment, only a simple linear transformation is performed on the local sample. Although the data volume of the plaintext after the linear transformation is larger than that before the linear transformation, the data volume of the plaintext after the linear transformation is still much smaller than that of the ciphertext. Therefore, compared with the horizontal federated learning modeling method based on data encryption methods such as homomorphic encryption or secret sharing, the amount of data transmitted between the participating devices and the federated server in this embodiment is less, and the computational workload of each participating device and the federated server in data calculation is less. This can improve the communication efficiency between the participating devices and the federated server in the horizontal federated learning modeling process, as well as the computational efficiency of each participating device and the federated server in data calculation in the horizontal federated learning modeling process.

[0084] As an example, the specific formula for performing periodic sample transformation on the local samples according to the private privacy protection module is as follows:

[0085]

[0086] Where O represents the periodically changing samples output by the private privacy protection module, W represents the neural network parameters, and X represents the local samples. Let be a periodic activation function with a period of 1 / r, and ε be the additional noise from the private noise module, as shown in the reference. Figure 2 , Figure 2 This is a schematic diagram illustrating the process of periodic sample transformation based on a private privacy protection module in this embodiment of the application. The input data X is the local sample, and the periodic neuron is activated by the following function: The periodic neural network has random noise ε as additional noise, and the output data O is the periodic transformation sample output by the private privacy protection module.

[0087] The horizontal federation model construction optimization method further includes:

[0088] Step A10: Obtain the sample to be predicted; according to the private privacy protection module, perform a periodic sample transformation on the sample to be predicted to obtain the first periodic transformation prediction sample.

[0089] Step A20: Based on the local shared neural network, perform sample prediction on the first periodic transformation prediction sample to obtain the first sample prediction result.

[0090] As an example, steps A10 to A20 include: obtaining a sample to be predicted, wherein the sample to be predicted consists of at least one feature value; inputting the sample to be predicted into a private privacy protection module to perform a periodic sample transformation on the local sample, so as to convert each feature value of the sample to be predicted into a feature value that conforms to periodic changes, thereby obtaining a first periodic transformation prediction sample; and performing sample prediction on the first periodic transformation prediction sample according to the local shared neural network to obtain a first sample prediction result, wherein the first sample prediction result may be a predicted label or a predicted probability value.

[0091] The horizontal federation model construction optimization method further includes:

[0092] Step B10: Receive the second periodic transformation prediction sample sent by the client, and perform sample prediction on the second periodic transformation prediction sample according to the local shared neural network to obtain the second sample prediction result. The second periodic transformation prediction sample is obtained by the client based on the target privacy protection module determined by the private privacy protection module issued by the participating device, and performs periodic sample transformation on the sample to be predicted.

[0093] Step B20: Feed back the prediction result of the second sample to the client.

[0094] Prior to the step of receiving the second periodic transformation prediction sample sent by the client, the horizontal federated model construction optimization method further includes:

[0095] In this embodiment, it should be noted that the participating device can deploy the private privacy protection module to the client. In this way, the client holds the private privacy protection module and the participating device holds the local shared neural network, which can realize secure inference between the two-party model between the client and the participating device.

[0096] As an example, steps B10 to B20 include: providing the private privacy protection module so that the client can use the private privacy protection module as the target privacy protection module, and performing periodic sample transformation on the private privacy protection module by inputting the sample to be predicted, so as to convert each feature value in the sample to be predicted into a periodic transformation feature value that conforms to periodic changes, thereby obtaining a second periodic transformation prediction sample; receiving the second periodic transformation prediction sample sent by the client, performing sample prediction on the second periodic transformation prediction sample according to the local shared neural network, and obtaining a second sample prediction result, which may be a predicted label or a predicted probability value; and feeding back the second sample prediction result to the client.

[0097] It should be noted that currently, in two-party model secure inference, participating devices are willing to provide the inference model as a service, but do not want to directly provide it to the client. The client wants to use the inference model to predict local samples, but considers local samples to be private information and does not want to transmit the plaintext of the local samples to the participating devices. Currently, the client typically transmits the homomorphically encrypted ciphertext of the local samples to the participating devices, and then the participating devices use the inference model to perform calculations on the homomorphically encrypted ciphertext in ciphertext mode. However, since the data volume of the homomorphically encrypted ciphertext is much larger than that of the plaintext data, the above-mentioned two-party model secure inference method based on homomorphic encryption will greatly increase the amount of communication data between the client and participating devices, as well as the amount of computation data for both the client and participating devices. This leads to lower communication efficiency between the client and participating devices, and lower computational efficiency for both the client and participating devices, thus affecting the efficiency of two-party model secure inference. In this embodiment, since the second periodic transformation prediction sample is subject to periodic changes, the sample feature value of the sample to be predicted corresponding to each periodic transformation feature value in the same second periodic transformation prediction sample is not unique. It is difficult for the participating device to deduce a unique sample to be predicted for the client based on the second periodic transformation prediction sample, thus protecting the client's data privacy. The participating device also does not need to provide the client with the local shared neural network used as the inference model. Furthermore, since the second periodic transformation prediction sample itself is plaintext data, the purpose of secure inference of the two-party model is achieved by exchanging plaintext data between the client and the participating device. The data volume of plaintext data is much smaller than that of ciphertext data. Therefore, it overcomes the technical defects of the two-party model secure inference method based on homomorphic encryption, which greatly increases the amount of communication data between the client and the participating device, as well as the amount of computation data of the client and the participating device, resulting in lower communication efficiency between the client and the participating device and lower computation efficiency of the client and the participating device. This improves the efficiency of secure inference of the two-party model.

[0098] Step C10: The private privacy protection module and the preset noise threshold are sent to the client so that the client can adjust the private noise module in the private privacy protection module according to the local additional noise determined by the preset noise threshold to obtain the target privacy protection module.

[0099] In this embodiment, it should be noted that the participating device can also send the private privacy protection module and the preset noise threshold to the client together. The client adjusts the private noise module in the private privacy protection module according to the local additional noise determined by the preset noise threshold to obtain the target privacy protection module. That is, the client obtains a target privacy protection module containing a private noise module with local additional noise. In this way, on the basis of converting the feature value of each sample in the sample to be predicted into a periodic transformation feature value that conforms to periodic changes, a local additional noise can be added to it. This local additional noise is held by the client alone, which can further increase the difficulty for the outside world to deduce the sample to be predicted from the client, thereby improving the data privacy protection effect in the secure inference of the two-party model in this embodiment. Moreover, since the local additional noise is limited to less than the preset noise threshold, it will not affect the prediction accuracy of the local shared neural network.

[0100] As an example, the local sample can be a local image sample, and the local shared neural network can be an image recognition neural network. Steps S10 to S50 include: acquiring local image samples; inputting the local image samples into a private privacy protection module to perform periodic sample transformation on the local image samples to obtain periodically transformed image samples; inputting the periodically transformed image samples into an image recognition neural network to perform local iterative training and optimization on the private privacy protection module and the image recognition neural network; acquiring the local network parameters of the image recognition neural network and uploading the local network parameters to a federated server, wherein the federated server is used to aggregate the local network parameters uploaded by each participating device into federated network parameters; receiving the federated network parameters issued by the federated server and updating the local network parameters of the image recognition neural network to the federated network parameters; returning to the execution step: acquiring local image samples, inputting the local image samples into a private privacy protection module to perform periodic sample transformation on the local image samples to obtain periodically transformed image samples, until the horizontal federated learning modeling is detected to be complete. In this way, since the periodically transformed image samples are obtained through periodic sample transformations, one periodically transformed image sample corresponds to the input image samples of multiple private privacy protection modules. Therefore, even if external parties deduce the periodically transformed image samples from the model parameters of the image recognition neural network, it is difficult to deduce which specific period's input image sample of the private privacy protection module the periodically transformed image sample corresponds to. The difficulty of deduction is equivalent to the difficulty of fault-tolerant learning. Thus, privacy protection of image data of participating devices is achieved in the process of building an image recognition model based on horizontal federated learning. Moreover, the local network parameters uploaded by participating devices to the federated server are all plaintext data. The data volume of plaintext data is much lower than that of homomorphically encrypted ciphertext data. Therefore, the amount of communication data transmitted between participating devices and the federated server in the process of building an image recognition model based on horizontal federated learning can be reduced, as can the amount of computation data calculated by each participating device and the federated server in the process of image data calculation. This improves the communication efficiency and image data calculation efficiency in the process of building an image recognition model based on horizontal federated learning, thereby improving the efficiency of building an image recognition model based on horizontal federated learning.

[0101] As an example, it should be noted that the private privacy protection module can be composed of one or more passport embedding network modules connected in series. The passport embedding network module can be composed of a private periodic neural network and a private noise module connected in series. The private privacy protection modules of each participating device can be heterogeneous networks; that is, the number of passport embedding network modules in the private privacy protection modules of each participating device can be different. Each participating device can design the number of passport embedding network modules in its own private privacy protection module according to its actual needs. For example, participating devices with a larger sample size can design a private privacy protection module composed of more passport embedding network modules to cope with more diverse data distributions. Complex sample data improves the accuracy of periodic sample transformation, thereby improving the accuracy of final sample prediction. For participating devices with a large number of samples, a private privacy protection module composed of fewer embedded network modules can be designed, reducing system resource consumption and improving the efficiency of periodic sample transformation and final sample prediction. Therefore, this application embodiment can adapt personalized heterogeneous private privacy protection modules to participating devices with different needs, meeting their individual requirements. That is, participating devices with a large number of samples need to adapt to private privacy protection modules with more complex network structures, while participating devices with a small number of samples need to adapt to private privacy protection modules with simpler network structures. See details below. Figure 3 , Figure 3 This is a schematic diagram illustrating the process of constructing a private privacy protection module and a local shared neural network based on horizontal federated learning in an embodiment of this application. N For local samples from participating devices, the private neural network D N As a private privacy protection module, the private neural network D N It consists of one or more passport embedding network modules. Each passport embedding network module can be composed of a private periodic neural network and a private noise module connected in series. The number of passport embedding network modules varies among the participating devices, therefore the private neural network D among the participating devices... N It can be a heterogeneous network, with a private periodic neural network used to perform periodic sample transformations on the training samples, a private noise module used to add noise to the output of the periodic privacy protection module, and a shared neural network G. N For locally shared neural networks, y~ N The predicted label output by the local shared neural network, y N For the sample labels of local samples, L N For model loss, G is sent by each participating device to the federated server. 1 To G N G is used to share local network parameters of the neural network in different participating devices.avg These are the parameters of the federal network.

[0102] This application provides a method for constructing and optimizing a horizontal federated model. In this horizontal federated learning process, each participating device is equipped with a local shared neural network and a private privacy protection module. The participating device can input local samples into the privacy protection module to perform periodic sample transformations on the local samples, obtaining periodically transformed samples. These periodically transformed samples are then input into the local shared neural network to perform local iterative training and optimization on the privacy protection module and the local shared neural network. The local network parameters of the local shared neural network are obtained and uploaded to a federated server. The federated server aggregates the local network parameters uploaded by each participating device into federated network parameters. The method receives the federated network parameters from the federated server and updates the local network parameters of the local shared neural network to the federated network parameters. The process then returns to the execution steps: obtaining local samples, inputting the local samples into the privacy protection module, and performing periodic sample transformations on the local samples to obtain periodically transformed samples, until the horizontal federated learning modeling is detected as complete. Since the periodic transformation samples are obtained through periodic sample transformation, one periodic transformation sample corresponds to the input samples of multiple private privacy protection modules. Therefore, even if the external system reverse-engineers the periodic transformation samples based on the model parameters of the local shared neural network, it is difficult to deduce which period's private privacy protection module's input sample the periodic transformation sample corresponds to. The difficulty of reverse-engineering is equivalent to the difficulty of fault-tolerant learning. Thus, data privacy protection for participating devices is achieved during the horizontal federated learning modeling process. Furthermore, the local network parameters uploaded by participating devices to the federated server are all plaintext data. Compared to the existing technology that uses homomorphic encryption for data privacy protection during the horizontal federated learning modeling process, the amount of plaintext data in this application is much smaller than that of homomorphically encrypted ciphertext data. Therefore, the amount of communication data transmitted between participating devices and the federated server during the horizontal federated learning modeling process can be reduced, improving the communication efficiency during the horizontal federated learning process. This solves the technical problem of how to improve the communication efficiency between participating devices during the horizontal federated learning modeling process while ensuring data privacy protection.

[0103] Example 2

[0104] This application provides a method for optimizing the construction of a horizontal federation model, applied to a federation server. The method for optimizing the construction of a horizontal federation model includes:

[0105] Step H10: Receive local network parameters uploaded by each participating device, and aggregate the local network parameters into federated network parameters. The local network parameters are obtained by the participating devices through iterative training and optimization of the local shared neural network and private privacy protection module based on periodic transformation samples. The periodic transformation samples are obtained by the participating devices through periodic sample transformation of local samples based on the private privacy protection module.

[0106] Step H20: The federated network parameters are sent to each of the participating devices, so that the participating devices can update the local network parameters of the local shared neural network to the federated network parameters, and return to the execution step: obtain local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodic transformed samples, until the horizontal federated learning modeling is detected to be completed.

[0107] In this embodiment, it should be noted that the aggregation method can be weighted average or weighted summation, etc. The participating device iteratively trains and optimizes the local shared neural network and private privacy protection module based on periodic sample transformation, performs periodic sample transformation on local samples based on the private privacy protection module, and updates the local network parameters of the local shared neural network to the federated network parameters. The specific implementation process can be referred to the specific content of steps S10 to S50 above, and will not be repeated here.

[0108] This application provides a method for optimizing the construction of a horizontal federated model. In this horizontal federated learning process, each participating device is equipped with a local shared neural network and a private privacy protection module. The participating device can input local samples into the privacy protection module to perform periodic sample transformations, obtaining periodically transformed samples. These periodically transformed samples are then input into the local shared neural network for local iterative training and optimization of the privacy protection module and the local shared neural network. The local network parameters of the local shared neural network are obtained and uploaded to a federated server. The federated server aggregates the local network parameters uploaded by each participating device into federated network parameters. The method receives the federated network parameters from the federated server and updates the local network parameters of the local shared neural network to these federated network parameters. The process then returns to the execution steps: obtaining local samples, inputting the local samples into the privacy protection module, and performing periodic sample transformations to obtain periodically transformed samples, until the horizontal federated learning modeling is detected as complete. Since the periodic transformation samples are obtained through periodic sample transformation, one periodic transformation sample corresponds to the input samples of multiple private privacy protection modules. Therefore, even if the external system reverse-engineers the periodic transformation samples based on the model parameters of the local shared neural network, it is difficult to deduce which period's private privacy protection module's input sample the periodic transformation sample corresponds to. The difficulty of reverse-engineering is equivalent to the difficulty of fault-tolerant learning. Thus, data privacy protection for participating devices is achieved during the horizontal federated learning modeling process. Furthermore, the local network parameters uploaded by participating devices to the federated server are all plaintext data. Compared to the existing technology that uses homomorphic encryption for data privacy protection during the horizontal federated learning modeling process, the amount of plaintext data in this application is much smaller than that of homomorphically encrypted ciphertext data. Therefore, the amount of communication data transmitted between participating devices and the federated server during the horizontal federated learning modeling process can be reduced, improving the communication efficiency during the horizontal federated learning process. This solves the technical problem of how to improve the communication efficiency between participating devices during the horizontal federated learning modeling process while ensuring data privacy protection.

[0109] Example 3

[0110] This application embodiment also provides a two-party security model inference method. The method of this embodiment is applied to a client, and the two-party security model inference method includes:

[0111] Step D10: Based on the target privacy protection module, perform periodic sample transformation on the sample to be predicted to obtain periodic transformed prediction samples. The target privacy protection module is determined by the private privacy protection module issued by the participating party device and a preset noise threshold.

[0112] Step D20: Upload the periodic transformation prediction sample to the participant device so that the participant device can perform sample prediction on the periodic transformation prediction sample based on the local shared neural network and obtain the sample prediction result.

[0113] Step D30: Receive the sample prediction results fed back by the participating device.

[0114] As an example, steps D10 to D30 include: inputting the sample to be predicted into the target privacy protection module, performing a periodic sample transformation on the sample to be predicted to convert the feature values ​​of each sample in the sample to be predicted into feature values ​​that conform to periodic changes, thereby obtaining a periodically transformed predicted sample, wherein the target privacy protection module is determined by the private privacy protection module issued by the participating device and a preset noise threshold; uploading the periodically transformed predicted sample to the participating device, so that the participating device can perform sample prediction on the periodically transformed predicted sample based on a local shared neural network to obtain a sample prediction result; and receiving the sample prediction result fed back by the participating device.

[0115] Prior to the step of performing periodic sample transformation on the sample to be predicted according to the target privacy protection module to obtain periodically transformed predicted samples, the horizontal federated model construction optimization method further includes:

[0116] Step E10: Receive the private privacy protection module and preset noise threshold sent by the participating device, and generate local additional noise according to the preset noise threshold;

[0117] Step E20: Adjust the private noise module in the private privacy protection module according to the local additional noise to obtain the target privacy protection module.

[0118] As an example, steps E10 to E20 include: receiving a private privacy protection module and a preset noise threshold from the participating device; generating local additional noise less than the preset noise threshold; adjusting the additional noise of the private noise module in the private privacy protection module to the local additional noise to obtain the target privacy protection module. Thus, the client obtains a target privacy protection module containing a private noise module with local additional noise. In this way, in addition to converting the feature value of each sample in the sample to be predicted into a periodic transformation feature value that conforms to periodic changes, a local additional noise can also be added to it. This local additional noise is held by the client alone, which can further increase the difficulty for the outside world to deduce the sample to be predicted from the client, thereby improving the data privacy protection effect in the secure inference of the two-party model in this application embodiment. Moreover, since the local additional noise is limited to less than the preset noise threshold, it will not affect the prediction accuracy of the local shared neural network.

[0119] This application provides a two-party model secure inference method, which involves performing periodic sample transformation on the sample to be predicted based on a target privacy protection module to obtain periodically transformed prediction samples. The target privacy protection module is determined by a private privacy protection module issued by the participating device and a preset noise threshold. The periodically transformed prediction samples are uploaded to the participating device, so that the participating device can perform sample prediction on the periodically transformed prediction samples based on a local shared neural network to obtain sample prediction results. The participating device then receives the sample prediction results fed back by the participating device. In this way, since the periodic transformation prediction sample is subject to periodic changes, the sample feature value of the sample to be predicted corresponding to each periodic transformation feature value in the same periodic transformation prediction sample is not unique. It is difficult for the participating device to infer a unique sample to be predicted based on the second periodic transformation prediction sample, thus protecting the client's data privacy. The participating device also does not need to provide the client with the local shared neural network used as the inference model. Furthermore, since the periodic transformation prediction sample itself is plaintext data, the purpose of secure two-party model inference is achieved by exchanging plaintext data between the client and the participating device. The data volume of plaintext data is much smaller than that of ciphertext data. Therefore, it overcomes the technical defects of two-party model secure inference methods based on homomorphic encryption, which greatly increases the amount of communication data between the client and the participating device, as well as the amount of computation data of the client and the participating device, resulting in lower communication efficiency between the client and the participating device and lower computation efficiency of the client and the participating device. This improves the efficiency of two-party model secure inference.

[0120] Example 4

[0121] Reference Figure 4This application also provides a horizontal federated learning system, which includes:

[0122] At least one participating device is used to acquire local samples, input the local samples into a private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, input the periodically transformed samples into a local shared neural network, and perform local iterative training and optimization on the private privacy protection module and the local shared neural network; acquire the local network parameters of the local shared neural network, and upload the local network parameters to the federated server; receive the federated network parameters issued by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters; return to the execution steps: acquire local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, until the lateral federated learning modeling is detected to be complete;

[0123] A federated server is used to aggregate the local network parameters uploaded by each of the participating devices into federated network parameters.

[0124] Optionally, the horizontal federated learning system also includes a client.

[0125] The participating device is also used to distribute the private privacy protection module to the client; receive the periodic transformation prediction sample sent by the client; perform sample prediction on the periodic transformation prediction sample according to the local shared neural network to obtain the sample prediction result; and feed back the sample prediction result to the client.

[0126] The client is used to perform periodic sample transformation on the sample to be predicted based on the target privacy protection module determined by the private privacy protection module, to obtain periodic transformed prediction samples; send the periodic transformed prediction samples to the participating device; and receive the sample prediction results fed back by the participating device.

[0127] Optionally, the private privacy protection module includes a private periodic neural network and a private noise module, and the participating device is further used for:

[0128] The local samples are input into the private periodic neural network, and periodic sample mapping is performed on the local samples to obtain periodic mapped samples.

[0129] Based on the proprietary noise module, noise is added to the periodic mapping sample to obtain the periodic transformation sample.

[0130] Optionally, the private periodic neural network includes neural network parameters and a periodic activation function, and the participating device is further configured to:

[0131] Based on the neural network parameters, the local samples are mapped to obtain mapped samples;

[0132] The periodic activation function is used to periodically activate the mapped sample to obtain the periodic mapped sample.

[0133] Optionally, the participating device is further used for:

[0134] Obtain the sample to be predicted, and perform a periodic sample transformation on the sample to be predicted according to the private privacy protection module to obtain the first periodic transformation prediction sample.

[0135] Based on the local shared neural network, sample prediction is performed on the first periodic transformation prediction sample to obtain the first sample prediction result.

[0136] Optionally, the participating device is further used for:

[0137] The private privacy protection module and the preset noise threshold are sent to the client.

[0138] Optionally, the client is also used for:

[0139] Based on the local additional noise determined by the preset noise threshold, the private noise module in the private privacy protection module is adjusted to obtain the target privacy protection module.

[0140] The horizontal federated learning system provided in this application adopts the horizontal federated model construction optimization method in the above embodiments, solving the technical problem of how to improve the communication efficiency between participating devices in the horizontal federated modeling process while protecting data privacy. Compared with the prior art, the beneficial effects of the horizontal federated learning system provided in this application are the same as those of the horizontal federated model construction optimization method provided in the above embodiments, and other technical features of this horizontal federated learning system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0141] Example 5

[0142] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the horizontal federation model construction optimization method in the first embodiment described above.

[0143] The following is for reference. Figure 5The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0144] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0145] Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0146] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0147] The electronic device provided in this application employs the horizontal federation model construction optimization method described in the above embodiments, solving the technical problem of how to improve the communication efficiency between participating devices during the horizontal federation modeling process while protecting data privacy. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the horizontal federation model construction optimization method provided in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0148] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0149] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0150] Example 6

[0151] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the method for optimizing the construction of the horizontal federation model in Embodiment 1 above.

[0152] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0153] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0154] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device causes the following: It acquires local samples; inputs the local samples into a private privacy protection module and performs periodic sample transformation on the local samples to obtain periodically transformed samples; inputs the periodically transformed samples into a local shared neural network and performs local iterative training and optimization on the private privacy protection module and the local shared neural network; acquires the local network parameters of the local shared neural network and uploads the local network parameters to a federated server, wherein the federated server is used to aggregate the local network parameters uploaded by each participating device into federated network parameters; receives the federated network parameters issued by the federated server and updates the local network parameters of the local shared neural network to the federated network parameters; and returns to the execution step: acquiring local samples and performing periodic sample transformation on the local samples by inputting the local samples into the private privacy protection module to obtain periodically transformed samples, until the completion of the lateral federated learning modeling is detected.

[0155] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0157] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0158] The computer-readable storage medium provided in this application stores computer-readable program instructions for executing the aforementioned horizontal federation model construction optimization method, solving the technical problem of how to improve the communication efficiency between participating devices in the horizontal federation modeling process while protecting data privacy. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application embodiment are the same as the beneficial effects of the horizontal federation model construction optimization method provided in the above embodiments, and will not be repeated here.

[0159] Example 7

[0160] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the horizontal federation model construction optimization method described above.

[0161] The computer program product provided in this application solves the technical problem of how to improve the communication efficiency between participating devices in the horizontal federation modeling process while protecting data privacy. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the horizontal federation model construction optimization method provided in the above embodiments, and will not be repeated here.

[0162] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. An optimization method for constructing a horizontal federation model, characterized in that, Applied to participating devices, the horizontal federation model construction optimization method includes: Local samples are obtained by inputting them into a private privacy protection module, which performs periodic sample transformation on the local samples to obtain periodically transformed samples. By inputting the periodic transformation samples into a local shared neural network, the private privacy protection module and the local shared neural network are subjected to local iterative training and optimization. The local network parameters of the local shared neural network are obtained, and the local network parameters are uploaded to the federation server. The federation server is used to aggregate the local network parameters uploaded by each of the participating devices into federation network parameters. Receive the federated network parameters sent by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters; Return to execution steps: Obtain local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, until the horizontal federated learning modeling is detected to be completed; The private privacy protection module includes a private periodic neural network and a private noise module. The private periodic neural network includes neural network parameters and a periodic activation function, wherein the periodic activation function is a periodic function. The step of inputting the local sample into the private privacy protection module and performing periodic sample transformation on the local sample to obtain periodically transformed samples includes: Based on the neural network parameters, the local samples are mapped to obtain mapped samples; The periodic activation function is used to periodically activate the mapped sample to obtain the periodic mapped sample. Based on the proprietary noise module, noise is added to the periodic mapping sample to obtain the periodic transformation sample.

2. The horizontal federation model construction optimization method as described in claim 1, characterized in that, The horizontal federation model construction optimization method also includes: Obtain the sample to be predicted, and perform a periodic sample transformation on the sample to be predicted according to the private privacy protection module to obtain the first periodic transformation prediction sample. Based on the local shared neural network, sample prediction is performed on the first periodic transformation prediction sample to obtain the first sample prediction result.

3. The horizontal federation model construction optimization method as described in any one of claims 1 to 2, characterized in that, The horizontal federation model construction optimization method also includes: The client receives a second periodic transformation prediction sample and performs sample prediction on the second periodic transformation prediction sample according to the local shared neural network to obtain a second sample prediction result. The second periodic transformation prediction sample is obtained by the client performing periodic sample transformation on the sample to be predicted based on the target privacy protection module determined by the private privacy protection module issued by the participating device. The prediction result of the second sample is fed back to the client.

4. The horizontal federation model construction optimization method as described in claim 3, characterized in that, Prior to the step of receiving the second periodic transformation prediction sample sent by the client, the horizontal federated model construction optimization method further includes: The private privacy protection module and the preset noise threshold are sent to the client so that the client can adjust the private noise module in the private privacy protection module according to the local additional noise determined by the preset noise threshold, thereby obtaining the target privacy protection module.

5. A horizontal federated learning system, characterized in that, The horizontal federated learning system includes: At least one participating device is used to acquire local samples, input the local samples into a private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, input the periodically transformed samples into a local shared neural network, and perform local iterative training and optimization on the private privacy protection module and the local shared neural network; acquire the local network parameters of the local shared neural network, and upload the local network parameters to the federated server; receive the federated network parameters issued by the federated server, and update the local network parameters of the local shared neural network to the federated network parameters; return to the execution steps: acquire local samples, input the local samples into the private privacy protection module, perform periodic sample transformation on the local samples to obtain periodically transformed samples, until the lateral federated learning modeling is detected to be complete; A federated server is used to aggregate the local network parameters uploaded by each of the participating devices into federated network parameters. The private privacy protection module includes a private periodic neural network and a private noise module. The private periodic neural network includes neural network parameters and a periodic activation function, wherein the periodic activation function is a periodic function. The participating device is further configured to: perform sample mapping on the local sample according to the neural network parameters to obtain a mapped sample; perform periodic activation on the mapped sample according to the periodic activation function to obtain the periodic mapped sample; and add noise to the periodic mapped sample according to the private noise module to obtain the periodic transformed sample.

6. The horizontal federated learning system as described in claim 5, characterized in that, The horizontal federated learning system also includes a client. The participating device is also used to distribute the private privacy protection module to the client; receive the periodic transformation prediction sample sent by the client; perform sample prediction on the periodic transformation prediction sample according to the local shared neural network to obtain the sample prediction result; and feed back the sample prediction result to the client. The client is used to perform periodic sample transformation on the sample to be predicted based on the target privacy protection module determined by the private privacy protection module, to obtain periodic transformed prediction samples; send the periodic transformed prediction samples to the participating device; and receive the sample prediction results fed back by the participating device.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the horizontal federation model construction optimization method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program implementing a horizontal federation model construction optimization method, the program implementing the horizontal federation model construction optimization method being executed by a processor to implement the steps of the horizontal federation model construction optimization method as described in any one of claims 1 to 4.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the horizontal federation model construction optimization method as described in any one of claims 1 to 4.

Images