A data aggregation method based on federated learning, electronic device and storage medium

Abstract

A data aggregation method based on federated learning, an electronic device and a storage medium belong to the technical field of federated learning. In order to improve the accuracy of data aggregation in the process of federated learning, the application comprises constructing a federated learning network: a federated learning network composed of a central server and a plurality of clients is constructed, each client has a local data set; selecting a client; the client trains; after the central server receives the parameters of the local neural network model uploaded by each client, the parameters of the global neural network model and the parameters of the local neural network model are used to construct a Gaussian distribution model, then the Gaussian center point of the Gaussian distribution model is calculated, the parameters of the neural network model of the Gaussian center point are set as the parameters of the global network model of the new round, and a new global neural network model is constructed according to the parameters, and then the accuracy is checked. The application has high accuracy and also has certain robustness.

Description

Technical Field

[0001] This invention belongs to the field of federated learning technology, specifically relating to a data aggregation method, electronic device, and storage medium based on federated learning. Background Technology

[0002] Federated learning technology, based on ensuring data privacy, security, and legal compliance, enables data sharing and collaborative modeling; it also addresses the problem of data silos to a certain extent and protects data security. Federated learning is a distributed machine learning framework designed to assist in training machine learning models while ensuring data privacy and security, achieving collaborative modeling and improving model performance. Federated learning primarily consists of a central server, participating parties, and local data. The training process of federated learning involves the following steps: 1. The central server establishes a basic model and distributes it to each participating party. 2. Each participating party trains its model using its own data and returns the trained model to the central server. 3. The central server aggregates the models returned by all participating parties to construct a more accurate model. This cycle of steps is repeated continuously, thereby improving the overall model's performance and effectiveness.

[0003] The performance of federated learning is determined by both the selected network model and the aggregation algorithm. The aggregation algorithm directly impacts the model aggregated by the central server; a poor aggregation algorithm can lead to a suboptimal federated model generated by the central server, preventing convergence. Therefore, it is evident that the aggregation algorithm has a significant impact on the final performance of the federated learning model.

[0004] To improve the accuracy of federated models, existing techniques typically optimize model precision by finding the centroid of the data. However, these methods often overlook the weight distribution of the data nodes, which can significantly influence centroid selection during the aggregation process. For example, when calculating the mean centroid, extreme values ​​can have a significant impact on the centroid; and when finding the geometric mean centroid, the centralized distribution of nodes in the dataset can significantly affect the geometric center. These factors can all lead to a decrease in the final model accuracy. Summary of the Invention

[0005] The problem this invention aims to solve is to improve the accuracy of data aggregation in the federated learning process, and proposes a data aggregation method, electronic device, and storage medium based on federated learning.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A data aggregation method based on federated learning includes the following steps:

[0008] S1. Constructing a Federated Learning Network: Construct a federated learning network consisting of a central server and multiple clients, with each client having its own local dataset;

[0009] S2. Client Selection: The central server selects clients from multiple clients. m Randomly select several clients S i The central server will set the parameters ω of the global neural network model. t Send to S i Where t = 1, 2, ..., T, t is any number of communications in T, and T is the number of communications required for the global neural network model to achieve an accuracy greater than or equal to a threshold after t communications; m = 1, 2, ..., M, M is the total number of clients in the constructed federated learning network; i = 1, 2, ..., k, k is the total number of randomly selected clients; where k < M; If it is the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω of the initialized neural network model. 0 As parameters of the global neural network model at t=1 communication;

[0010] S3. Client-side training: Several clients selected in step S2... i Receives parameters ω from the global neural network model of the central server. t A neural network model is constructed, and then trained using local data to obtain a local neural network model. The parameters of the trained local neural network model are then... Uploaded to the central server;

[0011] S4. Central Server Aggregates Data: The central server receives parameters from the local neural network models uploaded by each client. Then, using the parameters ω of the global neural network model t And the parameters of the local neural network model A Gaussian distribution model is constructed for the parameters, then the Gaussian center point of the Gaussian distribution model is calculated, and the parameters of the neural network model at the Gaussian center point are set as the parameters ω of the new global network model. t+1 Based on this, a new global neural network model is constructed;

[0012] S5. Accuracy Verification: The central server uses the local dataset to test the accuracy of the current global neural network model and compares it with a threshold. If the accuracy is less than the threshold, it returns to step S2 to continue training; if the accuracy is greater than or equal to the threshold, the learning ends.

[0013] Furthermore, the parameters ω of the new round of global network model described in step S4 t+1 The specific methods for determining this method include the following steps:

[0014] S4.1. Using the parameters ω of the global neural network model t And the parameters of the local neural network model The new Gaussian kernel function is constructed by using the parameters of the Gaussian kernel function, and its expression is as follows:

[0015]

[0016] Where σ is the width of the Gaussian kernel, The Gaussian kernel function;

[0017] S4.2. Calculate the Gaussian kernel function The center point of the Gaussian kernel function The parameters of the neural network model at the center point are set to the parameters ω of the new global network model. t+1 .

[0018] Furthermore, the specific implementation method of step S4.2 includes the following steps:

[0019] S4.2.1. For a randomly selected number of clients S i Gaussian kernel function The expression for summing values ​​is:

[0020]

[0021] Wherein, φ(ω) t ) is the objective function of the stochastic gradient ascent algorithm;

[0022] S4.2.2. Calculate φ(ω) using the stochastic gradient ascent method. t ) for ω t The gradient of is expressed as:

[0023]

[0024] in, This represents the total gradient of the objective function with respect to the global neural network parameters.

[0025] This allows us to obtain the parameters ω of the new global network model. t+1 ;

[0026] Let L be the number of iterations for stochastic gradient ascent, and let the formula for calculating stochastic gradient ascent be as follows:

[0027]

[0028] Where α is the step size of the stochastic gradient ascent; l = 1, 2, ..., L, where l is the number of any stochastic gradient ascent iteration in L; ω t,lLet ω be the global neural network model parameters on the current stochastic gradient. During the first iteration, ω... t,l =ω t ω is obtained by L iterations of stochastic gradient ascent. t,l+1 This refers to the parameters ω of the new global network model. t+1 .

[0029] Furthermore, an activation function is introduced into the formula for calculating stochastic gradient ascent to prevent the Gaussian kernel center from rising too rapidly near the center of the Gaussian kernel, thus missing the center point. The expression for the activation function is:

[0030]

[0031] in, For activation functions;

[0032] The formula for calculating stochastic gradient ascent with an activation function is as follows:

[0033]

[0034] Furthermore, α is defined as: 0.001 ≤ α ≤ 0.01.

[0035] An electronic device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the data aggregation method based on federated learning.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned data aggregation method based on federated learning.

[0037] The beneficial effects of this invention are:

[0038] The data aggregation method based on federated learning described in this invention utilizes the Gaussian distribution characteristic of the parameters of the neural network. In each round of federated learning data aggregation, the parameters ω of the global neural network model are used... t And the parameters of the local neural network model A Gaussian distribution model is constructed for the parameters. Then, the Gaussian center points of the Gaussian distribution model are calculated, and the parameters of the neural network model based on the Gaussian center points are used as the parameters ω of the new global network model. t+1 Based on this, a new global neural network model is constructed to aggregate data, improving the accuracy of data aggregation. Comparative experimental results show that the method of this invention has high accuracy and also has a certain degree of robustness. Attached Figure Description

[0039] Figure 1This is a flowchart of a data aggregation method based on federated learning as described in this invention;

[0040] Figure 2 A frequency histogram of the neural network weights generated for different nodes, which follows a Gaussian distribution.

[0041] Figure 3 A distribution diagram of the mean center and the Gaussian center;

[0042] Figure 4 Curves showing the summation of Gaussian kernel functions for normally distributed data points at different center points;

[0043] Figure 5 Activation function graph;

[0044] Figure 6 The diagram shows how the loss value of the method of the present invention changes with the number of iterations, where (a) shows how the loss value changes with the number of iterations in the MNIST dataset, and (b) shows how the loss value changes with the number of iterations in the CIFAR-10 dataset.

[0045] Figure 7 Accuracy comparison curves for different polymerization methods;

[0046] Figure 8 The robustness comparison curves for different polymerization methods;

[0047] Figure 9 This is pseudocode for a data aggregation method based on federated learning as described in this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0049] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0050] To further understand the invention's content, features, and effects, the following specific embodiments are listed, along with accompanying drawings. Figure 1 To be continued Figure 9 Detailed explanation is as follows:

[0051] Frequency histogram tests were performed on the parameters of 100 client-side neural networks trained in the same batch under the same neural network architecture. The mean of these data was used as the center (X0), and one-twentieth of the difference between the maximum and minimum values ​​was used as a step size (L). The frequency of the data appearing within each range was statistically analyzed, and a frequency histogram of these weights was plotted. Figure 2 As shown in the figure, these data approximately follow a normal distribution.

[0052] Then, a JB test was performed on these data, and the results are as follows:

[0053] Skewness: -0.07; Peak value: 2.78; JB statistic: 0.87

[0054] Therefore, all three results are within the acceptable range of the JB test data conforming to a normal distribution, indicating that these data conform to a normal distribution.

[0055] Finally, the KS test was performed on these data, and the p-value was 0.63, which is greater than the significance level of 0.05, proving that these data conform to a normal distribution.

[0056] It can be seen that the parameters of the neural networks trained in the same batch by clients under the same neural network architecture conform to a normal distribution, that is, they have Gaussian distribution characteristics. Therefore, this application proposes the following embodiments.

[0057] Example 1:

[0058] A data aggregation method based on federated learning, such as Figure 1 As shown, it includes the following steps:

[0059] S1. Constructing a Federated Learning Network: Building a network consisting of a central server and multiple client nodes. m The federated learning network consists of clients, each with its own local dataset; D m＝ {D1,D2,...,D M}, where m = 1, 2, ..., M, and M is the total number of clients in the federated learning network.

[0060] S2. Client Selection: The central server selects clients from multiple clients. m Randomly select several clients S i S i ={S1,S2,...,S} k}, where i = 1, 2, ..., k, and k is the total number of randomly selected clients. k < M; the central server will send the parameters ω of the global neural network model for the current communication round. t Send to randomly selected client S i Where t = 1, 2, ..., T, t is any number of communications in T, and T is the number of communications required for the accuracy of the global neural network model obtained after t communications to be greater than or equal to a threshold; if it is the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω of the initialized neural network model. 1 As parameters of the global neural network model at t=1 communication;

[0061] S3. Client-side training: Several clients selected in step S2... i Receives parameters ω from the global neural network model of the central server. t A neural network model is constructed, and then trained using local data to obtain a local neural network model. The parameters of the trained local neural network model are then... Uploaded to the central server;

[0062] S4. Central Server Aggregates Data: The central server receives parameters from the local neural network models uploaded by each client. Then, using the parameters ω of the global neural network model t And the parameters of the local neural network model A Gaussian distribution model is constructed for the parameters, and then the Gaussian center point of the Gaussian distribution model is calculated. Specifically, the center of a Gaussian distribution is usually its mean. For a Gaussian distribution, the mean is the center of symmetry of the curve in its probability density function (PDF). The mean is usually estimated by calculating the arithmetic mean of the samples, which is well known to those skilled in the art and will not be elaborated upon here. After obtaining the Gaussian center point of the Gaussian distribution model, the parameters of the neural network model at the Gaussian center point are set as the parameters ω of the new round of global network model. t+1 Based on this, a new global neural network model is constructed;

[0063] S5. Accuracy Verification: The central server uses the local dataset to test the accuracy of the current global neural network model and compares it with a threshold. If the accuracy is less than the threshold, it returns to S2 to continue training; if the accuracy is greater than or equal to the threshold, the learning ends.

[0064] The above uses the parameters ω of the global neural network model. t And the parameters of the local neural network model A Gaussian distribution model is constructed for the parameters, and the Gaussian center point of the Gaussian distribution model is calculated to determine the parameters ω of the new global network model. t+1 This improves the accuracy of federated learning data aggregation.

[0065] Because federated learning, in order to improve efficiency, randomly selects a number of clients from multiple clients for each model update, rather than selecting all clients, extreme values ​​can significantly affect the Gaussian center when calculating it; similarly, concentrated distribution of points can influence the geometric mean when searching for it. The impact of data distribution on the center point is as follows: Figure 3 As shown in the figure, the yellow dots represent 100 points that follow a Gaussian distribution, the blue dots represent the center point of the Gaussian distribution of these points, and the red dots represent the arithmetic mean of 50 points randomly selected from the 100 points. It can be clearly seen from the figure that the arithmetic mean point is significantly offset from the center point of the Gaussian distribution.

[0066] So, how do we find the center of the constructed Gaussian distribution model from randomly selected points, thereby better finding the parameters ω of the new global network model? t+1 Making the federal aggregation process more accurate is another technical problem that this application needs to solve.

[0067] First, generate a set of values ​​that follow the order x ~ N(μ,σ). 2 Normally distributed data points {x1, x2, ..., x} n}, select the Gaussian kernel function, the expression of which is:

[0068]

[0069] Where x is the independent variable of the Gaussian kernel function, ν is the center point of the Gaussian kernel function, and η is the hyperparameter of the Gaussian kernel function. For ease of analysis, η is set to a constant, and taken as... At this point, the Gaussian kernel function, expressed in equation (1.1), can be rewritten as follows:

[0070]

[0071] The n normally distributed data points generated by equation (1.2) are distributed on a Gaussian kernel function centered at ν. The summation of the n state distribution data points is expressed as follows:

[0072]

[0073] For each term in equation (1.3) at x i Performing a Taylor expansion at the point =ν, we get:

[0074]

[0075] in yes The smallest quantity of ν, and then taking the partial derivative of equation (1.4) with respect to ν, we can obtain:

[0076]

[0077] Taking the partial derivative of equation (1.5) with respect to ν, we can obtain:

[0078]

[0079] From the above, it can be seen that if we want to maximize the sum of the n normally distributed data points applied to the Gaussian kernel function centered at ν, then, At this point, the following needs to be satisfied:

[0080]

[0081] therefore, And because the data points {x1,x2,...,x} n} follows a normal distribution, therefore, Therefore, ν = μ, meaning that the center point of the Gaussian kernel function coincides with the center point of the normal distribution. In other words, the center point of the Gaussian kernel function coincides with the center point of the Gaussian distribution. Thus, the center point of the Gaussian distribution can be determined by finding the center point of the Gaussian kernel function, which means the center point of the Gaussian distribution model can be determined by finding the center point of the Gaussian kernel function. This solves the problem of correctly finding the center point of the Gaussian distribution model from randomly selected points and improves the accuracy of data aggregation.

[0082] The following simulation is performed to illustrate the above calculation process:

[0083] First, generate a set of data points {x1, x2, ..., xn} that follow a normal distribution from x to N(0,1). n}, take the Gaussian kernel function The interval for ν is chosen as ν∈[-1,1]. The data points are summed and plotted, as shown in Figure 4. The x-axis represents the center point of the Gaussian kernel, and the y-axis represents the sum of the normally distributed data points applied to the Gaussian kernel function. From... Figure 4 It can be seen that the summation of the Gaussian kernel function at different Gaussian kernel center points for normally distributed data points reaches its maximum when the center point of the Gaussian kernel function coincides with the center point of the Gaussian distribution. Figure 4 The results also indirectly demonstrate that when the number of samples is large, the remainder of the Taylor expansion of the Gaussian kernel function does not affect the overall conclusion.

[0084] The above results further demonstrate that the parameters of neural networks trained in the same batch under the same neural network architecture conform to a normal distribution, i.e., they exhibit Gaussian distribution characteristics. This also explains the rationale for determining the center point of the Gaussian distribution model by finding the center point of the Gaussian kernel function.

[0085] Furthermore, the center point of the Gaussian distribution model is determined by finding the center point of the Gaussian kernel function, thus solving the problem of correctly finding the center point of the Gaussian distribution model from randomly selected points. This relates to the parameters ω of the new round of global network model in step S4. t+1 The specific methods for determining this method include the following steps:

[0086] S4.1. Using the parameters ω of the global neural network model t And the parameters of the local neural network model As parameters of the Gaussian kernel function in equation (1.2), a new Gaussian kernel function is constructed, and its expression is as follows:

[0087]

[0088] Where σ is the width of the Gaussian kernel, The Gaussian kernel function;

[0089] S4.2. Calculate the Gaussian kernel function The center point of the Gaussian kernel function The center point is the Gaussian center point of the Gaussian distribution model, and the Gaussian kernel function. The parameters of the neural network model at the center point are the parameters ω of the new global network model. t+1 .

[0090] Furthermore, embodiments of this application also provide methods for calculating the Gaussian kernel function. The specific method for determining the center point, namely, the specific implementation method of step S4.2, includes the following steps:

[0091] S4.2.1. For a randomly selected number of clients S i Gaussian kernel function The expression for summing values ​​is:

[0092]

[0093] Wherein, φ(ω) t () is used as the objective function of the stochastic gradient ascent algorithm;

[0094] S4.2.2. Calculate φ(ω) using the stochastic gradient ascent method. t ) for ω t The gradient of is expressed as:

[0095]

[0096] in, The objective function is φ(ω) t For the global neural network parameters ω t The total gradient;

[0097] This allows us to obtain the parameters ω of the new global network model. t+1 ;

[0098] Specifically, let the number of stochastic gradient ascent iterations be L, where l is any iteration number in L, l = 1, 2, ..., L; the objective function of stochastic gradient ascent (1.7) can be transformed into:

[0099]

[0100] in, These are the parameters of the local neural network model; ω t,l represents the parameters of the global neural network model obtained in the l-th iteration; k represents the total number of randomly selected clients.

[0101] Then, the parameters ω of the global neural network model obtained in the l-th iteration are... t,l And the parameters of the local neural network model The Gaussian kernel function obtained by substituting into equation (1.6) is as follows:

[0102]

[0103] Substituting equation (1.10) into equation (1.8), the total gradient is:

[0104]

[0105] To ensure that the changes in the global neural network parameters occur at a reasonable rate, the coefficients of the total gradient are readjusted. Let α be the step size of the stochastic gradient ascent. Therefore, the formula for calculating the stochastic gradient ascent can be expressed as:

[0106]

[0107] Where α is the step size of the stochastic gradient ascent; ω t,l Let ω be the global neural network model parameters on the current stochastic gradient. During the first iteration, ω... t,l =ω t ω is obtained by L iterations of stochastic gradient ascent. t,l+1 This refers to the parameters ω of the new global network model. t+1 .

[0108] Furthermore, since the value of the Gaussian kernel function increases as it gets closer to the center of the Gaussian function, but the trend of change becomes more and more gradual, an activation function needs to be introduced to prevent the center of the Gaussian kernel function from rising too quickly when it gets close to the center of the Gaussian function, thus missing the center point.

[0109] An activation function is introduced into the formula for calculating stochastic gradient ascent to prevent the Gaussian kernel function from rising too quickly near the center of the Gaussian kernel, thus missing the center point. The mathematical expression of the activation function is as follows:

[0110]

[0111] in, The activation function is shown in the graph below. Figure 5 As shown in the figure, the activation function can adaptively adjust the step size α of the stochastic gradient ascent. The closer to the center, the smaller the step size, thus avoiding the situation where the ascent is too fast due to excessive compensation near the center, and thus misses the center point.

[0112] The formula for calculating stochastic gradient ascent with an activation function is as follows:

[0113]

[0114] Furthermore, the step size α directly affects the execution efficiency of stochastic gradient ascent. The smaller the value, the slower the execution speed, and the larger the value, the faster the execution speed. However, a large step size α often leads to the stochastic gradient ascent algorithm failing to converge during iteration. Therefore, the optimal range of α is: 0.001≤α≤0.01.

[0115] The simulation comparison test of the method in this embodiment is as follows:

[0116] The proposed scheme will be simulated using PyTorch according to the method of this embodiment, and its performance will be compared with that of similar algorithms such as FedAVG and Geometric. The simulation parameters are set as follows:

[0117] 1. Model: The network model is a three-layer feedforward neural network;

[0118] 2. Clients: The total number of clients is 100, and 10 clients are randomly selected to participate;

[0119] 3. Training parameters: Global training epochs 100, client-side local training epochs 5, client-side local training batch size 32, learning rate 0.01;

[0120] 4. Training data: MNIST dataset and CIFAR-10 dataset;

[0121] 5. The pseudocode for the test is as follows: Figure 9 As shown.

[0122] The test results of the algorithm's convergence and divergence are as follows Figure 6 As shown in the figure, the loss value of the aggregation method in this embodiment changes with the number of iterations. Figure 6 (a) shows the change of the loss value with the number of iterations in the MNIST dataset. It can be seen that the loss value eventually changes to about 0.017 after multiple iterations, and the change is small. Figure 6 (b) shows the change of the loss value with the number of iterations on the CIFAR-10 dataset. It can be seen that the loss value eventually changes to around 1.275 with a relatively small amplitude. This demonstrates that the method can converge after multiple iterations on both datasets.

[0123] The accuracy test results are as follows Figure 7 As shown, both the mean center and the geometric center can be significantly disturbed by the distribution of data, thus affecting accuracy. However, as the number of nodes increases, the impact of this disturbance caused by the data distribution gradually decreases. Figure 7 This demonstrates how accuracy changes with the number of nodes. (Through...) Figure 7 It is clear that the aggregation method proposed in this application has excellent performance. When finding the center point of data, the average center point is significantly affected by extreme data points and cannot accurately reflect the overall data center; the geometric center point is affected by concentrated data and also cannot accurately reflect the data center. However, the Gaussian center can reduce the influence of both of these factors, finding a more accurate data center point. Therefore, when using the aggregation method with the Gaussian center as the data center point, the accuracy of the trained neural network is consistently higher than that of the geometric mean and the federated mean.

[0124] Robustness test results such as Figure 8As shown in the figure, the performance of this aggregation method on the MNIST dataset is illustrated. The horizontal axis represents the proportion of Byzantine working points, and the vertical axis represents the accuracy after five iterations. It is clearly evident that the aggregation algorithm presented in this application has stronger robustness compared to the Fedavg algorithm, and its performance is slightly better than Fedavg even without attacks. Comparing it with the geometric mean algorithm, it can be seen from the figure that both aggregation algorithms have strong robustness. However, the accuracy of this aggregation algorithm is slightly higher than that of the geometric mean algorithm when Byzantine attacks are present. This is because when dealing with Byzantine attacks, the geometric center point is inevitably disturbed by toxic data, and the geometric mean algorithm cannot remove this toxic data, causing the calculated geometric center to deviate from the original geometric center, resulting in a decrease in accuracy. The aggregation method presented in this application, however, utilizes the filtering function of the Gaussian kernel function to filter out most of the toxic data, reducing the impact of toxic data on the overall training process. Therefore, the overall performance of the aggregation method presented in this application is better than that of the geometric mean algorithm. In summary, it can be seen that the aggregation method proposed in this application performs well under Byzantine attacks and has strong robustness.

[0125] Example 2:

[0126] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the data aggregation method based on federated learning described in Embodiment 1.

[0127] The computer device of the present invention may include a processor and a memory, such as a microcontroller containing a central processing unit. Furthermore, the processor executes the computer program stored in the memory to implement the steps of the aforementioned data aggregation method based on federated learning.

[0128] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0129] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0130] Example 3:

[0131] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned data aggregation method based on federated learning.

[0132] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the steps of the above-mentioned data aggregation method based on federated learning can be implemented.

[0133] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0134] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A data aggregation method based on federated learning, characterized in that, Includes the following steps: S1. Constructing a Federated Learning Network: Construct a federated learning network consisting of a central server and multiple clients, with each client having its own local dataset; S2. Client Selection: The central server selects clients from multiple clients. m Randomly select several clients S i The central server will set the parameters ω of the global neural network model. t Send to S i Where t = 1, 2, ..., T, t is any number of communications in T, and T is the number of communications required for the global neural network model to achieve an accuracy greater than or equal to a threshold after t communications; m = 1, 2, ..., M, M is the total number of clients in the constructed federated learning network; i = 1, 2, ..., k, k is the total number of randomly selected clients; where For the first communication, the central server initializes a neural network model as the global neural network model, and uses the parameters ω of the initialized neural network model. 0 As parameters of the global neural network model at t=1 communication; S3. Client-side training: Several clients selected in step S2... i Receives parameters ω from the global neural network model of the central server. t A neural network model is constructed, and then trained using local data to obtain a local neural network model. The parameters of the trained local neural network model are then... Uploaded to the central server; S4. Central Server Aggregates Data: The central server receives parameters from the local neural network models uploaded by each client. Then, using the parameters ω of the global neural network model t And the parameters of the local neural network model A Gaussian distribution model is constructed for the parameters, then the Gaussian center point of the Gaussian distribution model is calculated, and the parameters of the neural network model at the Gaussian center point are set as the parameters ω of the new global network model. t+1 Based on this, a new global neural network model is constructed; The parameters ω of the new round of global network model described in step S4 t+1 The specific methods for determining this method include the following steps: S4.

1. Using the parameters ω of the global neural network model t And the parameters of the local neural network model The new Gaussian kernel function is constructed by using the parameters of the Gaussian kernel function, and its expression is as follows: Where σ is the width of the Gaussian kernel, The Gaussian kernel function; S4.

2. Calculate the Gaussian kernel function The center point of the Gaussian kernel function The parameters of the neural network model at the center point are set to the parameters ω of the new global network model. t+1 ; The specific implementation method of step S4.2 includes the following steps: S4.2.

1. For a randomly selected number of clients S i Gaussian kernel function The expression for summing values ​​is: Where, φ(ω) t ) is the objective function of the stochastic gradient ascent algorithm; S4.2.

2. Calculate φ(ω) using the stochastic gradient ascent method. t ) for ω t The gradient of is expressed as: in, This represents the total gradient of the objective function with respect to the global neural network parameters. This allows us to obtain the parameters ω of the new global network model. t+1 ; Let L be the number of iterations for stochastic gradient ascent, and let the formula for calculating stochastic gradient ascent be as follows: Where α is the step size of the stochastic gradient ascent; l = 1, 2, ..., L, where l is the number of any stochastic gradient ascent iteration in L; ω t,l Let ω be the global neural network model parameters on the current stochastic gradient. During the first iteration, ω... t,l =ω t ω is obtained by L iterations of stochastic gradient ascent. t,l+1 This refers to the parameters ω of the new global network model. t+1 ; S5. Accuracy Verification: The central server uses the local dataset to test the accuracy of the current global neural network model and compares it with a threshold. If the accuracy is less than the threshold, it returns to step S2 to continue training; if the accuracy is greater than or equal to the threshold, the learning ends.

2. The data aggregation method based on federated learning according to claim 1, characterized in that, An activation function is introduced into the formula for calculating stochastic gradient ascent to prevent the Gaussian kernel center from rising too rapidly near the center of the Gaussian kernel, thus missing the center point. The expression for the activation function is: in, For activation functions; The formula for calculating stochastic gradient ascent with an activation function is as follows:

3. The data aggregation method based on federated learning according to claim 2, characterized in that, The value of α is: 0.001≤α≤0.

01.

4. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the data aggregation method based on federated learning as described in any one of claims 1-3.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a data aggregation method based on federated learning as described in any one of claims 1-3.

Images