Federal learning-based data aggregation method, electronic equipment and storage medium

Abstract

The invention discloses a federated learning-based data aggregation method, electronic equipment and a storage medium, and belongs to the technical field of federated learning. In order to improve the accuracy of data aggregation in the federated learning process, the method comprises the following steps: constructing a federated learning network: constructing the federated learning network consisting of a central server and a plurality of clients, each client having a local data set; selecting a client; the client carries out training; after receiving the parameters of the local neural network model uploaded by each client, the central server constructs a Gaussian distribution model by taking the parameters of the global neural network model and the parameters of the local neural network model as parameters, and then calculates a Gaussian center point of the Gaussian distribution model; parameters of the neural network model of the Gaussian center point are set as parameters of a new round of global network model, a new global neural network model is constructed according to the parameters, and then accuracy verification is carried out. The method has the advantages of high accuracy and certain robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning, and particularly relates to a data aggregation method, an electronic device, and a storage medium based on federated learning. Background Art

[0002] Based on ensuring data privacy security and legal compliance, federated learning technology realizes data sharing and joint modeling; and to a certain extent, it solves the problem of data islands and protects data security. Federated learning is a distributed machine learning framework, which is to assist in the model training of machine learning on the premise of ensuring data privacy security, realize joint modeling, and improve the model effect. Federated learning mainly consists of a central server, participants, and local data. The training process of federated learning is divided into the following steps: 1. The central server establishes a basic model and distributes the model to each participant. 2. Each participant uses its own data to train the model and returns the trained model to the central server. 3. The central server aggregates the models returned by each participant to construct a more accurate model. The cycle composed of the above steps is continuously repeated to improve the performance and effect of the overall model.

[0003] The performance of federated learning is jointly determined by the selected network model and the model aggregation algorithm. Among them, the aggregation algorithm directly affects the model aggregated by the central server. When the performance of the aggregation algorithm is poor, it may cause the central server to generate a very poor federated model, resulting in the model not converging. It can be seen from this that the aggregation algorithm has an important impact on the final performance of the federated learning model.

[0004] In order to improve the accuracy of the federated model, the prior art usually optimizes the model accuracy by finding the center point of the data. However, the existing methods often ignore the weight distribution of the data nodes, and the data weight distribution will affect the selection of the center point to a certain extent during the aggregation process. For example, when calculating the mean center, extreme values will have a greater impact on the center point; when finding the geometric mean point, the nodes distributed in the data set will have a significant impact on the geometric center. These factors will all lead to a decrease in the accuracy of the final model. Summary of the Invention

[0005] The problem to be solved by the present invention is to propose a data aggregation method, an electronic device, and a storage medium based on federated learning to improve the accuracy of data aggregation in the process of federated learning.

[0006] To achieve the above object, the present invention is realized through the following technical solutions:

[0007] A data aggregation method based on federated learning includes the following steps:

[0008] S1. Construct a federated learning network: Construct a federated learning network consisting of a central server and multiple clients, where each client has its own local dataset;

[0009] S2. Select clients: The central server randomly selects several clients S m from multiple clients D i , and the central server sends the parameters ω t of the global neural network model to S i , where t = 1, 2, …, T, t is any communication time in T, and T is the communication time when the accuracy of the global neural network model obtained after t communications is greater than or equal to the threshold; m = 1, 2,..., M, M is the total number of clients in the constructed federated learning network; i = 1, 2,..., k, k is the total number of randomly selected clients; where k < M; If it is the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω 0 of the initialized neural network model as the parameters of the global neural network model at t = 1 communication;

[0010] S3. Clients train: The several clients S i selected in step S2 receive the parameters ω t of the global neural network model from the central server, construct a neural network model, and then use the local data to train the constructed neural network model to obtain a local neural network model, and upload the parameters of the trained local neural network model to the central server;

[0011] S4. Central server aggregates data: After the central server receives the parameters of the local neural network models uploaded by each client, it constructs a Gaussian distribution model with the parameters ω t of the global neural network model and the parameters of the local neural network model, then calculates the Gaussian center point of the Gaussian distribution model, and sets the parameters of the neural network model at the Gaussian center point as the parameters ω t+1 of the new round of global network model, and constructs a new global neural network model accordingly;

[0012] S5. Accuracy verification: The central server uses the local dataset to test the accuracy of the current global neural network model and compares it with the threshold. If the accuracy is less than the threshold, it returns to step S2 for continued training; if the accuracy is greater than or equal to the threshold, the learning ends.

[0013] Furthermore, the method for determining the parameters ω t+1 of the new round of global network model in step S4 specifically includes the following steps:

[0014] S4.1. Use the parameters ω of the global neural network model t and the parameters of the local neural network model as the parameters of the Gaussian kernel function to construct a new Gaussian kernel function, and its expression is as follows:

[0015]

[0016] where σ is the Gaussian kernel width, is the Gaussian kernel function;

[0017] S4.2. Calculate the center point of the Gaussian kernel function , and set the parameters of the neural network model at the center point of the Gaussian kernel function as the parameters ω of the new round of global network model t+1 .

[0018] Furthermore, the specific implementation method of step S4.2 includes the following steps:

[0019] S4.2.1. Sum the values of the Gaussian kernel function i of several randomly selected clients S , and its expression is:

[0020]

[0021] where φ(ω t ) is the objective function of the stochastic gradient ascent algorithm;

[0022] S4.2.2. Use the method of stochastic gradient ascent to calculate the gradient of φ(ω t ) with respect to ω t , and its expression is:

[0023]

[0024] where is the total gradient of the objective function with respect to the global neural network parameters;

[0025] Thus, obtain the parameters ω of the new round of global network model t+1 ;

[0026] Set the number of iterations of stochastic gradient ascent to L, and set the calculation formula of stochastic gradient ascent as follows:

[0027]

[0028] where α is the step size of stochastic gradient ascent; l = 1, 2, …, L, and l is any iteration number of stochastic gradient ascent in L; ω t,lis the global neural network model parameter on the current random gradient. When the first round of iteration is executed, ω t,l = ω t ; ω t,l+1 obtained by performing random gradient ascent iteration L times is the parameter ω t+1 of the new round of global network model.

[0029] Furthermore, an activation function is introduced into the calculation formula of random gradient ascent to avoid the situation that the center of the Gaussian kernel function may rise too fast when approaching the Gaussian center and thus miss the center point. The expression of the activation function is:

[0030]

[0031] where is the activation function;

[0032] The calculation formula of random gradient ascent with the activation function introduced is as follows:

[0033]

[0034] Furthermore, the α is: 0.001 ≤ α ≤ 0.01.

[0035] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the data aggregation method based on federated learning are implemented.

[0036] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the data aggregation method based on federated learning is implemented.

[0037] Advantages of the present invention:

[0038] For the data aggregation method based on federated learning according to the present invention, based on the characteristic that the parameters of the neural network have a Gaussian distribution, when aggregating federated learning data in each round, a Gaussian distribution model is constructed with the parameter ω t of the global neural network model and the parameter of the local neural network model as parameters, and then the Gaussian center point of the Gaussian distribution model is calculated, and the parameter of the neural network model at the Gaussian center point is used as the parameter ω t+1 of the new round of global network model, and a new global neural network model is constructed accordingly to perform data aggregation, improving the accuracy of data aggregation. The comparison experiment results show that the method of the present invention has a high accuracy and also has a certain robustness. Description of the Drawings

[0039] Figure 1Flowchart of a data aggregation method based on federated learning according to the present invention;

[0040] Figure 2 Frequency histogram of neural network weights generated by different nodes following a Gaussian distribution;

[0041] Figure 3 Distribution diagram of the average center and Gaussian center;

[0042] Figure 4 Curve of the sum of Gaussian kernel functions of normal distribution data points at different center points;

[0043] Figure 5 Activation function diagram;

[0044] Figure 6 Variation of the loss value of the method of the present invention with the number of iterations. Among them, (a) shows the variation of the loss value with the number of iterations in the MNIST dataset, and (b) shows the variation of the loss value with the number of iterations in the CIFAR-10 dataset;

[0045] Figure 7 Accuracy comparison curve of different aggregation methods;

[0046] Figure 8 Robustness comparison curve of different aggregation methods;

[0047] Figure 9 Pseudocode of a data aggregation method based on federated learning according to the present invention. Detailed implementation manners

[0048] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. It should be understood that the specific implementation manners described herein are only used to explain the present invention and are not used to limit the present invention. That is, the specific implementation manners described are only a part of the implementation manners of the present invention, rather than all of the specific implementation manners. Usually, the components of the specific implementation manners of the present invention described and shown in the accompanying drawings herein can be arranged and designed in various different configurations, and the present invention can also have other implementation manners.

[0049] Therefore, the detailed description of the specific implementation manners of the present invention provided in the accompanying drawings below is not intended to limit the scope of the claimed present invention, but merely represents the selected specific implementation manners of the present invention. All other specific implementation manners obtained by those skilled in the art based on the specific implementation manners of the present invention without creative efforts belong to the scope of protection of the present invention.

[0050] To further understand the content, features, and effects of the present invention, the following specific embodiments are listed and described in detail in conjunction with the appended Figure 1 to the appended Figure 9 as follows:

[0051] Randomly select the parameters of the neural networks trained by 100 clients in the same batch under the same neural network architecture for frequency histogram testing. Use the average value of these data as the center (X 0 ), and use one-twentieth of the difference between the maximum and minimum values as a step size (L). Count the frequency of the data appearing in each range and draw a frequency histogram of the weights. As Figure 2 shown, it can be seen from the figure that these data approximately follow a normal distribution.

[0052] Then perform the J-B test on these data, and the results after the test are as follows:

[0053] Skewness: -0.07; Kurtosis: 2.78; J-B statistic: 0.87,

[0054] It can be seen that the above three results are all within the acceptable range of the J-B test for the data to conform to the normal distribution. Therefore, these data all conform to the normal distribution.

[0055] Finally, perform the K-S test on these data. The p-value after the test: 0.63, which is greater than the significance level of 0.05, also proving that these data conform to the normal distribution.

[0056] It can be seen that the parameters of the neural networks trained by the clients under the same neural network architecture in the same batch conform to the normal distribution, that is, they have Gaussian distribution characteristics. Therefore, the following embodiments are proposed in this application.

[0057] Embodiment 1:

[0058] A data aggregation method based on federated learning, as Figure 1 shown, includes the following steps:

[0059] S1. Construct a federated learning network: Construct a federated learning network composed of a central server and multiple clients D m , where each client has its own local dataset; D m＝ {D 1 , D 2 ,..., D M}, where m = 1, 2,..., M, and M is the total number of clients in the constructed federated learning network;

[0060] S2. Select clients: The central server randomly selects several clients S m from multiple clients D i , Si = {S 1 , S 2 ,..., S k}}, where i = 1, 2,..., k, k is the total number of randomly selected clients, k < M; The central server sends the parameters ω of the global neural network model in the current communication round t to the randomly selected client S i , where t = 1, 2,…, T, t is any communication time in T, and T is the number of communications when the accuracy of the global neural network model obtained after t communications is greater than or equal to the threshold; If it is the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω of the initialized neural network model 1 as the parameters of the global neural network model at t = 1 communication;

[0061] S3. The client conducts training: A number of clients S selected in step S2 i receive the parameters ω of the global neural network model from the central server t , construct a neural network model, and then use local data to train the constructed neural network model to obtain a local neural network model, and upload the parameters of the trained local neural network model to the central server;

[0062] S4. The central server aggregates data: After the central server receives the parameters of the local neural network models uploaded by each client , taking the parameters ω of the global neural network model t and the parameters of the local neural network model as parameters to construct a Gaussian distribution model, and then calculate the Gaussian center point of the Gaussian distribution model. Specifically, the center of the Gaussian distribution usually refers to its mean. For a Gaussian distribution, the mean is the symmetric center of the curve in its probability density function (PDF), and the mean is usually estimated by calculating the arithmetic mean of the samples, which is well known to those skilled in the art and will not be elaborated here. After obtaining the Gaussian center point of the Gaussian distribution model, set the parameters of the neural network model at the Gaussian center point as the parameters ω of the new round of global network model t+1 , and construct a new global neural network model accordingly;

[0063] S5. Accuracy verification: The central server uses the local dataset to test the accuracy of the current global neural network model and compares it with the threshold. If the accuracy is less than the threshold, return to S2 to continue training; If the accuracy is greater than or equal to the threshold, the learning ends.

[0064] The above, by using the parameters ω of the global neural network model tand the parameters of the local neural network model Construct a Gaussian distribution model for the parameters, and calculate the Gaussian center point of the Gaussian distribution model to determine the parameters ω of the new round of the global network model t+1 , improving the accuracy of federated learning data aggregation.

[0065] Because federated learning aims to improve efficiency, several clients are randomly selected from multiple clients each time for model update, rather than selecting all clients. In this way, when calculating the Gaussian center point, extreme values will have a greater impact on the Gaussian center; when finding the geometric mean point, points with a relatively concentrated distribution will affect the geometric center. The influence of data distribution on the center point is as Figure 3 shown. The yellow points represent 100 points subject to Gaussian distribution, the blue point represents the center point of the Gaussian distribution of these points, and the red point represents the arithmetic mean point randomly selected from 50 points out of 100 points. It can be clearly seen from the figure that the arithmetic mean point has a significant deviation compared with the center point of the Gaussian distribution.

[0066] So, how to find the center of the constructed Gaussian distribution model from the randomly selected points, so as to better find the parameters ω of the new round of the global network model t+1 , making the federated aggregation process more accurate, is another technical problem to be solved in this application.

[0067] First, generate a set of data points {x 2 , x 1 ,..., x 2} that follow the normal distribution x ∼ N(μ, σ n ), select the Gaussian kernel function, and the expression of the Gaussian kernel function is:

[0068]

[0069] where x is the independent variable of the Gaussian kernel function, ν is the center point of the Gaussian kernel function, and η is the hyperparameter of the Gaussian kernel function. For the convenience of analysis, η is set as a constant, and take At this time, the Gaussian kernel function, the expression form of Equation (1.1) is rewritten as:

[0070]

[0071] The n normal distribution data points generated by Equation (1.2) are distributed on the Gaussian kernel function centered on ν. Summing the n state distribution data points, the result is expressed as follows:

[0072]

[0073] Performing Taylor expansion on each term of Equation (1.3) at x i = ν, we can get:

[0074]

[0075] wherein is a minimal amount of, and taking the partial derivative of Equation (1.4) with respect to ν, we can obtain:

[0076]

[0077] Taking the partial derivative of Equation (1.5) with respect to ν, we can obtain:

[0078]

[0079] From the above, it can be seen that if it is required that the sum of the values of n generated normal distribution data points acting on the Gaussian kernel function centered at ν is the largest, then it is necessary to satisfy at this time:

[0080]

[0081] Therefore Also, because the data points {x 1 , x 2 ,..., x n} follow a normal distribution, therefore It can be obtained therefrom that ν = μ, that is, the center point of the Gaussian kernel function coincides with the center point of the normal distribution at this time, which means that the center point of the Gaussian kernel function coincides with the center point of the Gaussian distribution. Therefore, the center point of the Gaussian distribution can be determined by finding the center point of the Gaussian kernel function, that is, the center point of the Gaussian distribution model can be determined by finding the center point of the Gaussian kernel function, so as to correctly find the center point of the Gaussian distribution model from the randomly selected points and improve the accuracy of data aggregation.

[0082] For the above derivation process, the following simulation is carried out:

[0083] First, generate a set of data points {x 1 , x 2 ,..., x n} that follow the normal distribution x ~ N(0, 1), take the Gaussian kernel function and select the interval of ν as ν ∈ [-1, 1]. After summing the data points and plotting, as shown in Figure 4, where the x-axis is the center point of the Gaussian kernel and the y-axis is the sum of the values of the normal distribution data points acting on the Gaussian kernel function. It can be seen from Figure 4 that the sum of the Gaussian kernel functions of the normal distribution data points at different Gaussian kernel center points reaches the maximum when the center point of the Gaussian kernel function coincides with the center point of the Gaussian distribution. Through Figure 4The result also indirectly shows that when the number of samples is large, the remainder of the Taylor expansion of the Gaussian kernel function will not affect the overall conclusion.

[0084] The above results further prove that the parameters of the neural networks of clients under the same neural network architecture conform to the normal distribution during the same batch of training, that is, they have Gaussian distribution characteristics. It also shows the rationality of determining the center point of the Gaussian distribution model by finding the center point of the Gaussian kernel function.

[0085] Furthermore, by finding the center point of the Gaussian kernel function to determine the center point of the Gaussian distribution model, it is possible to correctly find the center point of the Gaussian distribution model from randomly selected points, and the parameters ω of the new round of global network model in step S4 t+1 The determination method specifically includes the following steps:

[0086] S4.1. Using the parameters ω of the global neural network model t and the parameters of the local neural network model as the parameters of the Gaussian kernel function in formula (1.2), a new Gaussian kernel function is constructed, and its expression is as follows:

[0087]

[0088] where σ is the Gaussian kernel width, is the Gaussian kernel function;

[0089] S4.2. Calculate the center point of the Gaussian kernel function , the center point of the Gaussian kernel function is the Gaussian center point of the Gaussian distribution model, and the parameters of the neural network model of the center point of the Gaussian kernel function are the parameters ω of the new round of global network model t+1 .

[0090] Furthermore, the embodiment of the present application also gives a specific method for calculating the center point of the Gaussian kernel function , that is, the specific implementation method of step S4.2 includes the following steps:

[0091] S4.2.1. Sum the values of the Gaussian kernel functions i of several randomly selected clients S , and its expression is:

[0092]

[0093] where φ(ω t ) is used as the objective function of the stochastic gradient ascent algorithm;

[0094] S4.2.2. Use the method of stochastic gradient ascent to calculate φ(ω t)The gradient of ω t is expressed as:

[0095]

[0096] where is the total gradient of the objective function φ(ω t ) with respect to the global neural network parameters ω t ;

[0097] Thus, the parameters ω t+1 of the new round of the global network model are obtained;

[0098] Specifically, let the number of random gradient ascent iterations be L, and l be any random gradient ascent iteration in L, l = 1, 2, …, L; the objective function of random gradient ascent, Equation (1.7), can be transformed into:

[0099]

[0100] where are the parameters of the local neural network model; ω t,l are the parameters of the global neural network model obtained in the l-th iteration; k is the total number of randomly selected clients.

[0101] Then, substituting the parameters ω t,l of the global neural network model obtained in the l-th iteration and the parameters of the local neural network model into Equation (1.6), the Gaussian kernel function is as follows:

[0102]

[0103] The total gradient obtained by substituting Equation (1.10) into Equation (1.8) is:

[0104]

[0105] To make the change of the global neural network parameters vary within a reasonable range, the coefficient of the total gradient is reset to α, where α is the step size of random gradient ascent. Therefore, the calculation formula of random gradient ascent can be expressed as:

[0106]

[0107] where α is the step size of random gradient ascent; ω t,l are the parameters of the global neural network model in the current random gradient ascent. When performing the first iteration, ω t,l = ω t ; the ω t,l+1 obtained after L random gradient ascent iterations is the parameter ω of the new round of the global network modelt+1 。

[0108] Furthermore, since the value of the Gaussian kernel function increases as it gets closer to the center of the Gaussian function, but the rate of change becomes increasingly gentle, it is necessary to introduce an activation function to prevent the problem that the center of the Gaussian kernel function may rise too quickly and miss the center point when approaching the Gaussian center. Therefore, an activation function is introduced to prevent the problem that the center of the Gaussian kernel function may rise too quickly and miss the center point when approaching the Gaussian center.

[0109] An activation function is introduced into the calculation formula of stochastic gradient ascent to avoid the situation that the center of the Gaussian kernel function may rise too quickly and miss the center point when approaching the Gaussian center. The mathematical expression of the activation function is as follows:

[0110]

[0111] where is the activation function, and its graph is as shown in Figure 5 It can be seen from the figure that the activation function can adaptively adjust the step size α of stochastic gradient ascent. The closer to the center position, the smaller the step size, thus avoiding the situation that the ascent is too fast due to excessive compensation near the center and missing the center point;

[0112] The calculation formula of stochastic gradient ascent with the introduction of the activation function is as follows:

[0113]

[0114] Furthermore, the size of the step size α value directly affects the execution efficiency of stochastic gradient ascent. The smaller the value, the slower the execution speed; the larger the value, the faster the execution speed. However, a relatively large step size α value often leads to non-convergence during the iteration of the stochastic gradient ascent algorithm. Therefore, the optimal value range of α is: 0.001 ≤ α ≤ 0.01.

[0115] The method of this embodiment is subjected to simulation comparison tests as follows:

[0116] The above scheme will be simulated using the method PyTorch of this embodiment, and the performance of the method of this embodiment will be compared with that of the same type of FedAVG algorithm and Geometric algorithm. The simulation parameter settings are as follows:

[0117] 1. Model: The network model is a three-layer feedforward neural network;

[0118] 2. Clients: The total number of clients is 100, and 10 clients are randomly selected to participate;

[0119] 3. Training parameters: 100 global training epochs, 5 local training epochs for each client, local training batch size of 32 for each client, and learning rate of 0.01;

[0120] 4. Training data: MNIST dataset and CIFAR-10 dataset;

[0121] 5. The pseudo-code for testing is as Figure 9 shown.

[0122] The test results of the algorithm's convergence and divergence are as Figure 6 shown. The figure shows the variation of the loss value of the aggregation method in this embodiment with the number of iterations. Figure 6 (a) shows the variation of the loss value with the number of iterations in the MNIST dataset. It can be seen that the loss value finally varies around 0.017 after multiple iterations, and the variation range is small. Figure 6 (b) shows the variation of the loss value with the number of iterations in the CIFAR-10 dataset. It can be seen that the loss value finally varies around 1.275 and the range is small. Thus, it can be seen that this method can converge after multiple rounds of iteration in both datasets.

[0123] The test results of the accuracy are as Figure 7 shown. Both the average center and the geometric center will have large perturbations due to the data distribution, which will affect the accuracy. As the number of nodes increases, the influence of the perturbations brought by this data distribution gradually decreases. Figure 7 It shows the variation of the accuracy with the number of nodes. Through Figure 7 it can be clearly seen that the aggregation method of this application has good performance. Because when finding the center point of the data, the average center point is greatly affected by extreme data and cannot well reflect the overall data center, and the geometric center point is affected by concentrated data and also cannot well reflect the data center. However, the Gaussian center can reduce the influence of these two situations and find a more accurate data center point. Therefore, when using the aggregation method with the Gaussian center as the data center point, the accuracy of the trained neural network is always higher than that of the geometric mean and the federated mean.

[0124] The test results of the robustness are as Figure 8As shown, the performance of this aggregation method in the MNIST dataset is presented. The horizontal axis represents the proportion of Byzantine working points, and the vertical axis represents the accuracy after five iterations. It can be intuitively seen that the aggregation algorithm of this application is more robust than the Fedavg algorithm, and in the absence of attacks, its performance is also slightly better than that of Fedavg. When compared with the geometric mean algorithm, it can be seen from the figure that both aggregation algorithms are quite robust. However, the accuracy of this aggregation algorithm in the presence of Byzantine attacks is slightly higher than that of the geometric mean algorithm. This is because when dealing with Byzantine attacks, the geometric center point will inevitably be disturbed by the poisonous data, and the geometric mean algorithm cannot eliminate these poisonous data, resulting in the calculated geometric center deviating from the original geometric center and causing a decrease in accuracy; while the aggregation method of this application uses the screening function of the Gaussian kernel function to filter out most of the poisonous data and reduce the impact of poisonous data on the overall training process. Therefore, the overall performance of the aggregation method of this application is better than that of the geometric mean algorithm. In summary, it can be seen that the aggregation method of this application has good performance and strong robustness under Byzantine attacks.

[0125] Embodiment 2:

[0126] An electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of a data aggregation method based on federated learning described in Embodiment 1 are implemented.

[0127] The computer device of the present invention may be a device including a processor and a memory, such as a single-chip microcomputer including a central processing unit. Moreover, when the processor is used to execute the computer program stored in the memory, the steps of the above-mentioned data aggregation method based on federated learning are implemented.

[0128] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0129] The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0130] Embodiment 3:

[0131] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the data aggregation method based on federated learning described above.

[0132] The computer-readable storage medium of the present invention can be any form of storage medium readable by the processor of the computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. A computer program is stored on the computer-readable storage medium. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the data aggregation method based on federated learning described above can be implemented.

[0133] The computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0134] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

[0135] Although the present application has been described above with reference to specific embodiments, various improvements can be made thereto and components thereof can be replaced with equivalents without departing from the scope of the present application. In particular, as long as there is no structural conflict, the various features in the specific embodiments disclosed in the present application can be combined with each other in any way, and the reason for not exhaustively describing the situations of these combinations in this specification is only to save space and resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A data aggregation method based on federated learning, characterized in that: The steps include: S1. Build a federated learning network: Build a federated learning network consisting of a central server and multiple clients, each of which has a local dataset; S2. Client selection: The central server selects clients from multiple clients D m Randomly select several clients S i , the central server sets the parameters ω of the global neural network model t Send to S i , where t = 1, 2, ..., T, t is the number of any communication in T, T is the number of communications when the accuracy of the global neural network model obtained after t communications is greater than or equal to the threshold; m = 1, 2, ..., M, M is the total number of clients of the constructed federated learning network; i = 1, 2, ..., k, k is the total number of randomly selected clients; where k<M; if it is the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω of the initialized neural network model 0 As the parameters of the global neural network model at t = 1 communication; S3. Client training: Several clients S selected in step S2 i Receive the parameters ω of the global neural network model from the central server t , build a neural network model, then use local data to train the built neural network model to obtain a local neural network model, and set the parameters of the trained local neural network model Upload to the central server; S4. Central server aggregates data: The central server receives the parameters of the local neural network model uploaded by each client. Then, the parameters ω of the global neural network model are t and the parameters of the local neural network model Construct a Gaussian distribution model for the parameters, then calculate the Gaussian center point of the Gaussian distribution model, and set the parameters of the neural network model of the Gaussian center point as the parameters ω of the new round of global network model t+1 , and based on this, a new global neural network model is constructed; S5. Accuracy check: The central server uses the local data set to test the accuracy of the current global neural network model and compares it with the threshold. If the accuracy is less than the threshold, it returns to step S2 to continue training; if the accuracy is greater than or equal to the threshold, the learning ends.

2. A data aggregation method based on federated learning according to claim 1, characterized in that: The parameter ω of the new round of global network model in step S4 is t+1 The determination method specifically includes the following steps: S4.

1. Taking the parameters ω of the global neural network model t and the parameters of the local neural network model As the parameter of the Gaussian kernel function, a new Gaussian kernel function is constructed, and its expression is as follows: Where σ is the Gaussian kernel width, is the Gaussian kernel function; S4.

2. Calculation of Gaussian kernel function The center point of the Gaussian kernel function The parameters of the neural network model at the center point are set as the parameters of the new round of global network model ω t+1 .

3. A data aggregation method based on federated learning according to claim 2, characterized in that: The specific implementation method of step S4.2 includes the following steps: S4.2.

1. For several randomly selected clients S i Gaussian kernel function The value is summed, and the expression is: Among them, φ(ωt) is the objective function of the stochastic gradient ascent algorithm; S4.2.

2. Use the stochastic gradient ascent method to calculate φ(ωt) versus ω t The gradient of is expressed as: in, is the total gradient of the objective function with respect to the global neural network parameters; Thus, the parameters ω of the new round of global network model are obtained t+1 ; Set the number of stochastic gradient ascent iterations to L, and set the calculation formula of stochastic gradient ascent as follows: Where α is the step size of stochastic gradient ascent; l = 1, 2, …, L, l is the number of random gradient ascent iterations in L; ω t,l is the global neural network model parameter on the current stochastic gradient. When executing the first round of iteration, ω t,l =ω t ; ω obtained by stochastic gradient ascent L times t,l+1 That is the parameter ω of the new round of global network model t+1 .

4. The data aggregation method based on federated learning according to claim 3, characterized in that: The activation function is introduced into the calculation formula of stochastic gradient ascent to prevent the center of the Gaussian kernel function from rising too fast and missing the center point when it approaches the Gaussian center. The expression of the activation function is: in, is the activation function; The calculation formula of stochastic gradient ascent with the activation function is as follows:

5. A data aggregation method based on federated learning according to claim 3 or 4, characterized in that: The α is: 0.001≤α≤0.

01.

6. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of a data aggregation method based on federated learning as described in any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data aggregation method based on federated learning described in any one of claims 1 to 5 is implemented.

Images