Federal learning method and architecture based on dynamic sparse pulse neural network optimization

Abstract

The invention relates to the crossing field of a federated learning algorithm framework, dynamic sparse training and a spiking neural network, and discloses a federated learning method and architecture for dynamic sparse spiking neural network optimization, and the method comprises the steps: employing a cloud edge end hierarchical architecture, and carrying out the training of local data on an edge device through a spiking neural network; the global model and the sparse mask received by the edge device generate a local sparse model for training; after each round of local training, the edge device uploads new model parameters to the edge server; after the edge server receives the model parameters, the model parameters are aggregated in combination with information such as historical contribution degree and the like, and are uploaded to the central cloud server; on the cloud server, the model parameters are further aggregated to generate a global model, and the global model is broadcasted to each edge server and each edge device before the next round of training. The sparse training and the spiking neural network are introduced into federated learning, so that the computing communication overhead is remarkably reduced, and the performance of the global model is improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed machine learning technology, and to the intersection of a federated learning algorithm framework, dynamic sparse training, and a spiking neural network. More specifically, it relates to a federated learning method and architecture based on dynamic sparse spiking neural network optimization. Background Art

[0002] Federated learning, as a distributed learning method, can aggregate local training models of multiple edge devices into a global model while protecting data privacy, thereby reducing the communication burden and privacy risks of centralized training. However, traditional federated learning methods are usually based on dense neural networks, with high computational complexity and high energy consumption. In addition, there is a large communication pressure on resource-constrained edge devices, making it difficult to operate efficiently.

[0003] Spiking neural networks simulate the spiking activity of biological neurons and adopt an event-driven computing model, which significantly reduces computational complexity and energy consumption, making them a potential efficient model on edge devices. However, SNNs still face the challenge of maintaining accuracy and reducing communication costs in resource-constrained environments.

[0004] In addition, dynamic sparse training reduces the complexity of training and reasoning by reducing the parameters in the neural network and optimizing the computational efficiency. However, traditional sparse training methods may increase computational and memory consumption while reducing the model size, which limits their application on edge devices.

[0005] Therefore, how to effectively combine these technologies to improve learning efficiency on edge devices remains the current research focus. Summary of the invention

[0006] The present invention provides a federated learning method based on dynamic sparse pulse neural network optimization, aiming to reduce the communication overhead of federated learning while ensuring the accuracy of the model.

[0007] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0008] A federated learning method based on dynamic sparse spiking neural network optimization, characterized by comprising the following steps:

[0009] Step 1: Using a cloud-edge-device layered architecture, local data is trained on edge devices using a spiking neural network and sparsified based on the neuron activation pattern of the time step;

[0010] Step 2: The edge device generates a local sparse model based on the received global model and sparse mask, and trains the local sparse model using a local data set;

[0011] Step 3: After each round of local training, the edge device uploads the updated model parameters to the corresponding edge server;

[0012] Step 4: After receiving the model parameters of multiple edge devices, the edge server aggregates them based on historical contribution information and uploads the aggregated model parameters to the central cloud server.

[0013] Step 5: On the central cloud server, the model parameters of all edge servers are further aggregated to generate a global model, which is broadcast to each edge server before the next round of training, and each edge server broadcasts to the edge devices.

[0014] In this technical solution, by directly and dynamically training the sparse pulse neural network model in the process of federated learning, the amount of calculation in the training process is reduced, the communication cost between devices is reduced, the training process is accelerated, and the computational communication overhead of federated learning is significantly reduced; in addition, an adaptive dynamic weighted aggregation method based on historical contribution is introduced in the process of federated learning. According to the historical training contribution of the client, the update weight of the global model is adaptively adjusted, which reduces the model instability caused by data heterogeneity, accelerates the convergence speed of the global model, and effectively reduces the communication overhead, thereby enhancing the performance of the model under non-independent and identically distributed data.

[0015] As a preferred solution, the sparseness is performed according to the neuron activation pattern of the time step in each training cycle, including:

[0016] At each time step, the edge device measures the neuron’s membrane potential V i (t) is calculated, and its expression is:

[0017]

[0018] Where δ is the attenuation factor of membrane potential, W ij represents the connection weight between neurons i and j, and S j (t)∈{0,1} indicates whether neuron j emits a pulse at time t;

[0019] When the membrane potential V i (t) exceeds the threshold V th When , the neuron fires a spike and resets the membrane potential;

[0020] According to the activation frequency of neurons, the local model is sparsely constructed through sparse masking to cover up the connections of inactive neurons and reduce unnecessary computation and transmission burden.

[0021] As a preferred solution, the local model is sparsely processed by a sparse mask according to the neuron activation frequency, specifically:

[0022] The model parameters are sparsely constructed using a sparse mask, where S k (t) Mask the inactive neuron connections to zero, and the sparsification model parameter expression is:

[0023]

[0024] In the formula, θ k (t) is the original local model parameters of client k at time step t, are the local model parameters after applying the sparse mask at time step t, S k (t)∈{0,1} is a sparse mask matrix, and inactive neuron connections are masked to zero.

[0025] As a preferred solution, the edge device generates a local sparse model according to the received global model and sparse mask, specifically:

[0026] By using the received global model and sparse mask, the local model parameters are sparsified, and the Hadamard inner product is performed to generate a sparse sub-network adapted to the local data distribution.

[0027] As a preferred solution, the method of using a local data set to train a local sparse model includes:

[0028] Use local data sets to train sparse models, capture data features by calculating the average firing frequency of neurons, and adjust the activation frequency of neurons based on these features to achieve an efficient training process;

[0029] Based on local training data and dynamic sparse training strategy, network connections are dynamically adjusted according to the sparsity of the model during training.

[0030] As a possible design of this preferred solution, the local training data and dynamic sparse training strategy are specifically:

[0031] Adopted Sparse distribution strategy, which allocates sparsity according to the parameter size of the layer;

[0032] According to the instantaneous gradient information of the local sparse model, the same number of neurons and connections with the largest gradients are added and removed to restore the sparsity of the model to the original sparsity S.

[0033] As a possible design of this preferred solution, the network connection is dynamically adjusted according to the sparsity of the model during the training process, specifically:

[0034] Design an adjustable hyperparameter α to dynamically adjust the sparse connection, and use the cosine annealing strategy to update the α value using the following formula:

[0035]

[0036] Among them, α0 is the initial sparsity control value, t is the current training step number, and T is the total training step number.

[0037] As a preferred solution, after receiving the model parameters of multiple edge devices, the edge server aggregates the model parameters in combination with information such as historical contribution, including:

[0038] In each round of training, the edge server receives the sparse model parameters uploaded by the client and calculates the sparse model parameters based on the historical contribution C of each client. i , determine the weighted ratio w i , whose expression is:

[0039]

[0040] Where N is the number of clients;

[0041] Each client's historical contribution C i Through its gradient norm Evaluate and dynamically adjust based on the decay factor λ to ensure that the contribution of recent training rounds has a higher weight. The expression is:

[0042]

[0043] The weighted average strategy is used to aggregate the model parameters of each client. In the process of updating the global model, an intermediate global model θ is first calculated by the sparse weighted average method. t+1 / 2 [k], its expression is:

[0044]

[0045] in, is the kth sparse parameter uploaded by client i in round t, w i is the weight of client i, M i is the corresponding sparse mask, and then the server applies the pruning strategy to optimize the intermediate global model to obtain the final global model θ t+1 .

[0046] A federated learning architecture based on dynamic sparse pulse neural network optimization includes a cloud server, an edge server and an edge device, wherein the cloud server is connected to the edge server, and the edge server is connected to the edge device;

[0047] The cloud server is used to send the global model and mask to the edge server, and aggregate the local sparse models uploaded by the edge server to update the global model; the mask is generated based on the sparsity, indicating whether each parameter in the global model should be retained;

[0048] The edge server is used to receive and forward the global model and mask to the edge device, and transmit the local sparse model uploaded by the edge device back to the cloud server for aggregation;

[0049] The edge device receives the global model and the mask, trains the local sparse model using the local data set, updates the local model, and then uploads the updated local sparse model to the edge server.

[0050] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0051] The present invention adopts a dynamic sparse pulse neural network training method in the process of federated learning, which significantly reduces the computing and communication overhead. At the same time, it introduces an adaptive dynamic weighted aggregation method based on historical contribution, which not only takes into account the local data size of the client, but also dynamically reflects the changes in the contribution of the client during the entire training process, especially in a federated learning environment where the data distribution is not independent and identically distributed, thereby improving the robustness and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Flowchart of the sparse training method for federated learning;

[0053] Figure 2 This is a schematic diagram of the cloud-edge hierarchical federated learning framework;

[0054] Figure 3 It is a schematic diagram of the framework of the federated learning method based on dynamic sparse spiking neural network optimization of the present invention;

[0055] Figure 4 This is a comparison chart of the accuracy results of testing the federated learning method based on dynamic sparse spiking neural network optimization in Example 3 and other federated learning methods on the MNIST dataset. DETAILED DESCRIPTION

[0056] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0057] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0058] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0059] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0060] Example 1

[0061] This embodiment provides a federated learning method based on dynamic sparse spiking neural network optimization, see Figure 1 ,include:

[0062] Step 1: Using a cloud-edge-device layered architecture, local data is trained on edge devices using a spiking neural network and sparsified based on the neuron activation pattern of the time step;

[0063] Step 2: The edge device generates a local sparse model based on the received global model and sparse mask, and trains the local sparse model using a local data set;

[0064] Step 3: After each round of local training, the edge device uploads the updated model parameters to the corresponding edge server;

[0065] Step 4: After receiving the model parameters of multiple edge devices, the edge server aggregates them based on historical contribution information and uploads the aggregated model parameters to the central cloud server.

[0066] Step 5: On the central cloud server, the model parameters of all edge servers are further aggregated to generate a global model, which is broadcast to each edge server before the next round of training, and each edge server broadcasts to the edge devices.

[0067] In this embodiment, by directly and dynamically training the sparse pulse neural network model in the process of federated learning, the amount of computation in the training process is reduced, the communication cost between devices is reduced, the training process is accelerated, and the computational communication overhead of federated learning is significantly reduced; in addition, an adaptive dynamic weighted aggregation method based on historical contribution is introduced in the process of federated learning. According to the historical training contribution of the client, the update weight of the global model is adaptively adjusted, which reduces the model instability caused by data heterogeneity, accelerates the convergence speed of the global model, and effectively reduces the communication overhead, thereby enhancing the performance of the model under non-independent and identically distributed data.

[0068] In a preferred embodiment, sparsification is performed according to the neuron activation pattern of the time step in each training cycle, including:

[0069] At each time step, the edge device measures the neuron’s membrane potential V i (t) is calculated, and its expression is:

[0070]

[0071] Where δ is the attenuation factor of membrane potential, W ij represents the connection weight between neurons i and j, and S j (t)∈{0,1} indicates whether neuron j emits a pulse at time t;

[0072] When the membrane potential V i (t) exceeds the threshold V th When , the neuron fires a spike and resets the membrane potential;

[0073] According to the activation frequency of neurons, the local model is sparsely constructed through sparse masking to cover up the connections of inactive neurons and reduce unnecessary computation and transmission burden.

[0074] In a preferred embodiment, the local model is sparsely processed by a sparse mask according to the neuron activation frequency, specifically:

[0075] The model parameters are sparsely constructed using a sparse mask, where S k (t) Mask the inactive neuron connections to zero, and the sparsification model parameter expression is:

[0076]

[0077] In the formula, θ k (t) is the original local model parameters of client k at time step t, are the local model parameters after applying the sparse mask at time step t, S k (t)v{0,1} is a sparse mask matrix where inactive neuron connections are masked to zero.

[0078] In a preferred embodiment, the edge device generates a local sparse model according to the received global model and sparse mask, specifically:

[0079] By using the received global model and sparse mask, the local model parameters are sparsified, and the Hadamard inner product is performed to generate a sparse sub-network adapted to the local data distribution.

[0080] In a preferred embodiment, the training of the local sparse model using the local data set includes:

[0081] Use local data sets to train sparse models, capture data features by calculating the average firing frequency of neurons, and adjust the activation frequency of neurons based on these features to achieve an efficient training process;

[0082] Based on local training data and dynamic sparse training strategy, network connections are dynamically adjusted according to the sparsity of the model during training.

[0083] In an optional embodiment, the local training data and dynamic sparse training strategy are specifically:

[0084] Adopted -Rényi-Kernel sparse distribution strategy, which allocates sparsity according to the parameter size of the layer;

[0085] According to the instantaneous gradient information of the local sparse model, the same number of neurons and connections with the largest gradients are added and removed to restore the sparsity of the model to the original sparsity S.

[0086] In an optional embodiment, the network connection is dynamically adjusted according to the sparsity of the model during the training process, specifically:

[0087] Design an adjustable hyperparameter α to dynamically adjust the sparse connection, and use the cosine annealing strategy to update the α value using the following formula:

[0088]

[0089] Among them, α0 is the initial sparsity control value, t is the current training step number, and T is the total training step number.

[0090] In a preferred embodiment, after receiving the model parameters of multiple edge devices, the edge server aggregates the model parameters in combination with information such as historical contribution, including:

[0091] In each round of training, the edge server receives the sparse model parameters uploaded by the client and calculates the sparse model parameters based on the historical contribution C of each client. i , determine the weighted ratio w i , whose expression is:

[0092]

[0093] Where N is the number of clients;

[0094] Each client's historical contribution C i Through its gradient norm Evaluate and dynamically adjust based on the decay factor λ to ensure that the contribution of recent training rounds has a higher weight. The expression is:

[0095]

[0096] The weighted average strategy is used to aggregate the model parameters of each client. In the process of updating the global model, an intermediate global model θ is first calculated by the sparse weighted average method. t+1 / 2 [k], its expression is:

[0097]

[0098] in, is the kth sparse parameter uploaded by client i in round t, w i is the weight of client i, Mi is the corresponding sparse mask, and then the server applies the pruning strategy to optimize the intermediate global model to obtain the final global model θ t+1 .

[0099] Example 2

[0100] This implementation proposes a cloud-edge hierarchical federated learning architecture. Figure 2 , applied to the federated learning method based on dynamic sparse pulse neural network optimization proposed in Example 1, including a cloud server, an edge server and an edge device, wherein the cloud server is connected to the edge server, and the edge server is connected to the edge device;

[0101] The cloud server is used to send the global model and mask to the edge server, and aggregate the local sparse models uploaded by the edge server to update the global model; the mask is generated based on the sparsity, indicating whether each parameter in the global model should be retained;

[0102] The edge server is used to receive and forward the global model and mask to the edge device, and transmit the local sparse model uploaded by the edge device back to the cloud server for aggregation;

[0103] The edge device receives the global model and the mask, trains the local sparse model using the local data set, updates the local model, and then uploads the updated local sparse model to the edge server.

[0104] Example 3

[0105] This example uses the public MNIST dataset to experiment with the federated learning method based on dynamic sparse spiking neural network optimization proposed in Example 1. Figure 1-Figure 4 .

[0106] The MNIST dataset (Mixed National Institute of Standards and Technology database) is a large handwritten digit database collected and compiled by the National Institute of Standards and Technology of the United States, containing a training set of 60,000 examples and a test set of 10,000 examples.

[0107] Consider a typical federated learning framework: the global model is set as a convolutional neural network containing two 5*5 convolutional layers, two maximum pooling layers, and two fully connected layers; this design introduces a pulse neural network into the classic federated learning framework, processes input data through time steps and sparse activation mechanisms, and uses the pulse emission characteristics to optimize the communication and computing efficiency of the model, while maintaining the privacy protection characteristics of federated learning and improving the adaptability and performance of edge devices in low-power scenarios. The design has a total of 400 local clients, and 20 local clients are randomly selected in each communication round to participate in the training. Each local client uses the SGD optimizer to iterate 10 times on the local dataset in each round and communicates with the server 10 times.

[0108] As a non-limiting example, during the training process, the server sets the sparsity S = 0.2, initializes the global model, sets the mask according to the sparsity, randomly selects 20 local clients, and sends the global model and mask to the selected local clients; after the local client receives the global model and mask, it generates a local sparse model, trains the local model on the local data set, inputs the local data x into the local sparse model in small batches of 32 samples, and the local sparse model performs prediction. The learning rate η is preset to 0.01 and the local sparse model is updated.

[0109] After every ten rounds of training, the local client will make a mask adjustment to dynamically update the structure of the sparse network. Specifically, set α = 0.01. When the local training is completed, the local client communicates with the server to remove the connections between some neuron nodes in the local sparse model, thereby increasing the sparsity of the local sparse model to a higher value. Subsequently, based on the instantaneous gradient information of the local sparse model, the same number of neurons and the connections with the largest gradients are added and removed to restore the sparsity of the local sparse model to S. According to the cosine decay update plan Adjust the change of sparsity. Where α0 is the initial sparsity control value, t is the current training step number, and T is the total training step number.

[0110] After the local client selected in this round completes the training and update of the local sparse model, it uploads the local model to the server. The server uses dynamic aggregation and combines historical contribution information, including:

[0111] In each round of training, the edge server receives the sparse model parameters uploaded by the client and calculates the sparse model parameters based on the historical contribution C of each client. i , determine the weighted ratio w i , whose expression is:

[0112]

[0113] Where N is the number of clients;

[0114] Each client's historical contribution C i Through its gradient norm Evaluate and dynamically adjust based on the decay factor λ to ensure that the contribution of recent training rounds has a higher weight. The expression is:

[0115]

[0116] The weighted average strategy is used to aggregate the model parameters of each client. In the process of updating the global model, an intermediate global model θ is first calculated by the sparse weighted average method. t+1 / 2 [k], its expression is:

[0117]

[0118] in, is the kth sparse parameter uploaded by client i in round t, w i is the weight of client i, M i is the corresponding sparse mask, and then the server applies the pruning strategy to optimize the intermediate global model to obtain the final global model θ t+1 .

[0119] After the edge server completes the aggregation of the local sparse models uploaded by the 20 local clients participating in this round of training, it sends the newly generated global model to the selected local clients and starts a new round of communication training until the global model converges.

[0120] In addition, this embodiment also selects a convolutional neural network with the same structure and the same settings as the above global model to perform the MNIST classification prediction task. 20 local clients are selected from 400 local clients. Under the condition of given sparsity S = 0.2, each local client uses the SGD optimizer to iterate 10 times on the local data set each round, communicates with the server 10 times, and performs prediction after federated learning training. The accuracy of the prediction results is as follows: Figure 3 As shown, it is obvious that compared with FedAvg, FedDST and FedSNN-Avg, the model performance obtained by the federated learning method based on dynamic sparse pulse neural network optimization proposed in this embodiment is better, and higher accuracy can be obtained with less communication cost.

[0121] Example 4

[0122] This paper proposes a federated learning system based on dynamic sparse spiking neural network optimization. Figure 3 , applied to the federated learning method based on dynamic sparse spiking neural network optimization proposed in Example 1, including spiking neural network and dynamic sparse training;

[0123] The spiking neural network is deployed on edge and terminal devices. By introducing the spiking neural network into the federated learning framework, the timing characteristics of the spiking neural network can be fully utilized, thereby improving the model's ability to process time series data and capture dynamic changes. This time series processing capability can not only improve the adaptability of the model, but also reduce the computational burden and improve computational efficiency.

[0124] The dynamic sparse training is performed by Sparse distribution, dynamically adjusts the connection sparse structure of the model during the training process, and combines pruning and regrowth mechanisms to enable the model to maintain efficient sparse distribution at different levels.

[0125] The same or similar reference numerals correspond to the same or similar components;

[0126] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0127] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A federated learning method based on dynamic sparse spiking neural network optimization, characterized in that: The steps include: Step 1: Using a cloud-edge-device layered architecture, local data is trained on edge devices using a spiking neural network and sparsified based on the neuron activation pattern of the time step; Step 2: The edge device generates a local sparse model based on the received global model and sparse mask, and trains the local sparse model using a local data set; Step 3: After each round of local training, the edge device uploads the updated model parameters to the corresponding edge server; Step 4: After receiving the model parameters of multiple edge devices, the edge server aggregates them based on historical contribution information and uploads the aggregated model parameters to the central cloud server. Step 5: On the central cloud server, the model parameters of all edge servers are further aggregated to generate a global model, which is broadcast to each edge server before the next round of training, and each edge server broadcasts to the edge devices.

2. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 1, characterized in that: The step 1 in each training cycle is to perform sparsification according to the neuron activation pattern of the time step, including: At each time step, the edge device measures the neuron’s membrane potential V i (t) is calculated, and its expression is: Where δ is the attenuation factor of membrane potential, W ij represents the connection weight between neurons i and j, and S j (t)∈{0,1} indicates whether neuron j emits a pulse at time t; When the membrane potential V i (t) exceeds the threshold V th When , the neuron fires a spike and resets the membrane potential; The local model is sparsely mapped using a sparse mask based on the neuron activation frequency.

3. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 2, characterized in that: The local model is sparsely constructed by using a sparse mask according to the neuron activation frequency, specifically: The model parameters are sparsely constructed using a sparse mask, where S k (t) Mask the inactive neuron connections to zero, and the sparsification model parameter expression is: In the formula, θ k (t) is the original local model parameters of client k at time step t, are the local model parameters after applying the sparse mask at time step t, S k (t)v{0,1} is a sparse mask matrix where inactive neuron connections are masked to zero.

4. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 1, characterized in that: The edge device in step 2 generates a local sparse model based on the received global model and sparse mask, specifically: By using the received global model and sparse mask, the local model parameters are sparsified, and the Hadamard inner product is performed to generate a sparse sub-network adapted to the local data distribution.

5. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 1, characterized in that: The local sparse model is trained using the local dataset as described in step 2, including: Use local data sets to train sparse models, capture data features by calculating the average firing frequency of neurons, and adjust the activation frequency of neurons based on these features to achieve an efficient training process; Based on local training data and dynamic sparse training strategy, network connections are dynamically adjusted according to the sparsity of the model during training.

6. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 5, characterized in that: The local training data and dynamic sparse training strategy are specifically: Adopted Sparse distribution strategy, which allocates sparsity according to the parameter size of the layer; According to the instantaneous gradient information of the local sparse model, the same number of neurons and connections with the largest gradients are added and removed to restore the sparsity of the model to the original sparsity S.

7. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 5, characterized in that: The network connection is dynamically adjusted according to the sparsity of the model during the training process, specifically: Design an adjustable hyperparameter α to dynamically adjust the sparse connection, and use the cosine annealing strategy to update the α value using the following formula: Among them, α0 is the initial sparsity control value, t is the current training step number, and T is the total training step number.

8. The method of federated learning based on dynamic sparse spiking neural network optimization according to claim 1, characterized in that: After receiving the model parameters of multiple edge devices in step 4, the edge server aggregates the model parameters based on historical contribution information, including: In each round of training, the edge server receives the sparse model parameters uploaded by the client and calculates the sparse model parameters based on the historical contribution C of each client. i , determine the weighted ratio w i , whose expression is: Where N is the number of clients; Each client's historical contribution C i Through its gradient norm G i t Evaluate and dynamically adjust based on the decay factor λ to ensure that the contribution of recent training rounds has a higher weight. The expression is: The weighted average strategy is used to aggregate the model parameters of each client. In the process of updating the global model, an intermediate global model θ is first calculated by the sparse weighted average method. t+1 / 2 [k], its expression is: in, is the kth sparse parameter uploaded by client i in round t, w i is the weight of client i, M i is the corresponding sparse mask, and then the server applies the pruning strategy to optimize the intermediate global model to obtain the final global model θ t+1 .

9. A federated learning architecture based on dynamic sparse pulse neural network optimization, comprising a cloud server, an edge server and an edge device, wherein the cloud server is connected to the edge server, and the edge server is connected to the edge device; characterized in that: The cloud server is used to send the global model and mask to the edge server, and aggregate the local sparse models uploaded by the edge server to update the global model; the mask is generated based on the sparsity, indicating whether each parameter in the global model should be retained; The edge server is used to receive and forward the global model and mask to the edge device, and transmit the local sparse model uploaded by the edge device back to the cloud server for aggregation; The edge device receives the global model and the mask, trains the local sparse model using the local data set, updates the local model, and then uploads the updated local sparse model to the edge server.

Images