Federal learning method for visual module of edge device
By using convolutional pulse neural network in the edge device vision module, local training and parameter aggregation are used for ignition rate and membrane potential characteristic screening equipment, the SNN network federated learning solution has high communication cost, low accuracy and slow convergence speed. Problem, achieving more efficient model training and accuracy.
Patent Information
- Application Number
- CN202411190638.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The federated learning scheme based on SNN network in the edge device vision module has problems such as high communication cost, low model accuracy and slow convergence speed.
Using convolutional pulse neural network, the ignition rate and average membrane potential are calculated locally on edge devices, and high-efficiency devices are screened for local training and parameter aggregation based on these characteristics, and reset potential and time constants are adjusted to improve the accuracy and convergence speed of the model. .
It improves the convergence speed and model accuracy of federated learning, reduces communication costs and training rounds, and is suitable for image recognition model training of heterogeneous edge device vision modules.
Smart Images

Figure CN119089981A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a federated learning technology, and in particular to a federated learning method for a visual module of an edge device. Background Art
[0002] The edge device vision module can identify and capture objects by capturing, processing and analyzing images, providing support for subsequent machine vision-based decision-making. With the development of IoT technology, the application of edge device vision modules has become more and more popular, such as smart home devices, and the privacy leakage problem has gradually become more prominent and more important. The privacy data protection of edge device vision modules mainly faces two challenges: on the one hand, users do not want their private data to be leaked, and require the right to know and control the use of their data; on the other hand, device manufacturers hope to collect as much user data as possible to provide better smart services.
[0003] Therefore, how to effectively protect the privacy data of users in the visual module of edge devices has become an important issue that needs to be solved urgently. Federated learning can better solve the privacy problem of data. Federated learning is a variant of distributed learning that aims to protect data privacy while allowing multiple edge devices to collaboratively train models, so that the models have better adaptability. The main process of federated learning is: the server initializes the global model and sends it to the edge devices participating in federated learning. Each edge device uses its own stored data to train the model locally and transmits the training parameters to the server. The server aggregates the parameters uploaded by each edge device, updates the parameters of the global model, and distributes them to each edge device again. Each edge device then trains the new global model locally, and repeats this cycle. Finally, the trained global model is obtained and distributed to each edge device participating in federated learning.
[0004] There are many types of edge devices, and the communication capabilities, communication stability, and local computing power of different devices are different, presenting complex and diverse heterogeneous forms. During the federated training process, edge devices will lose communication content or disconnect due to their own limitations, thus affecting the overall effect of the federated learning system. Therefore, how to solve the problem of variable terminals in the federated learning system is very important for the application and implementation of edge device vision modules such as smart home IoT systems.
[0005] Currently, there are related studies on federated learning systems with variable terminal devices, including artificial neural networks (ANN) and spiking neural networks (SNN), as follows:
[0006] (1) In terms of ANN, asynchronous federated learning and knowledge distillation are usually used to solve the problem of variable terminal devices in federated learning. Asynchronous federated learning allows terminal devices to complete local model updates at different times to adapt to the uncertainty of variable terminal devices. Distillation learning transfers the knowledge of the global model to the local model to promote better learning of the local model, thereby solving the problem of terminal devices receiving too little effective information due to variable terminal devices.
[0007] (2) In terms of SNN, by combining the more computationally sparse and simple spiking neural units with federated learning, a federated learning system with lower power consumption and lower computing power requirements than the ANN model method is realized, making it adaptable to a wider variety of terminal devices. These methods improve the model training and model aggregation methods in the federated learning system to adapt it to the spiking neural network, and also study the accuracy and robustness of the SNN network in the federated learning system.
[0008] For the asynchronous federated learning method of ANN, although it can alleviate the impact of terminal variability to a certain extent, the model parameters updated by each device may have an unstable impact on other devices, resulting in the overall convergence speed of the federated learning system being relatively slow, and it is difficult to tune and analyze the convergence and model performance. For the knowledge distillation method of ANN, there is a problem of information loss, because the knowledge of the global model is compressed and transferred to the local model, which may not fully express the complexity of the global model. At the same time, if the knowledge distillation process is improper, it may cause the local model to be overly dependent on the specific features of the global model, resulting in serious overfitting problems.
[0009] Regarding the combination of SNN and federated learning, although current relevant research shows that this method can achieve better power consumption performance than ANN, enabling SNN to cope with more complex and resource-constrained edge devices, the SNN model under the federated learning system still has problems such as high communication cost, low model accuracy, and slow convergence speed. Summary of the invention
[0010] The technical problem to be solved by the present invention is to propose a federated learning method for the visual module of an edge device, so as to solve the problems of high communication cost, low model accuracy and slow convergence speed existing in the federated learning scheme based on SNN network for the visual module of edge devices such as smart home edge devices.
[0011] The technical solution adopted by the present invention to solve the above technical problems is:
[0012] A federated learning method for an edge device vision module, wherein the edge device vision module includes a convolutional spiking neural network, wherein the convolutional spiking neural network includes a convolutional network and a spiking layer thereof and a fully connected layer and a spiking layer thereof; the training of the convolutional spiking neural network includes the following steps:
[0013] A1. Each edge device participating in this round of training performs local calculations according to the following steps:
[0014] A11. The edge device receives the global model sent by the server and updates its local model.
[0015] A12. Extract a local sample from the local sample set of the edge device and input the sample into the local model of the edge device.
[0016] A13. After the local model of the edge device completes the calculation of T time steps, the firing rate is calculated based on the pulse emission of each spiking neuron in the spiking layer of its fully connected layer at each time step t; T is a preset hyperparameter representing the time step of the convolutional spiking neural network;
[0017] A14, the edge device uploads its ignition rate to the server;
[0018] A2. Select edge devices and perform local training according to the following steps:
[0019] A21. Filter the top K devices with the highest firing rate from each edge device 1 edge devices, forming the training device set of this round, the K 1 is an integer ≥ 2;
[0020] A22. For each edge device in the training device set, use its local data set and local model to perform local training according to a preset number of local training rounds to obtain its updated local model parameters;
[0021] A23. Each edge device in the training device set uploads its updated local model parameters to the server respectively.
[0022] A3. The server aggregates the updated local model parameters of each edge device in the received training device set, and updates the global model based on the aggregation results;
[0023] A4. Determine whether the preset completion condition is met. If so, stop training. The global model at this time is the convolutional pulse neural network that has completed training; otherwise, return to step A1 to perform the next round of global training.
[0024] Furthermore, step A13 further includes: after the local model of the edge device completes the calculation of T time steps, calculating the average membrane potential of the edge device based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer;
[0025] In step A14, the edge device uploads its firing rate and average membrane potential to the server;
[0026] Before step A21 of step A2, the method further includes: performing mean calculation based on the average membrane potential of each edge device to obtain the global average membrane potential of this round of training; based on the global average membrane potential, screening and obtaining the top K with the largest difference with it. 2 edge devices; for the K obtained by screening 2 The edge devices respectively use the difference between their average membrane potential and the global average membrane potential to update their reset potential and time constant; the K 2 is an integer ≥2.
[0027] Furthermore, the reset potential and time constant are updated according to the following formula:
[0028]
[0029]
[0030] Among them, V k2 is the average membrane potential of the k2th edge device, x k2 is the difference between the average membrane potential of the k2th edge device and the global average membrane potential, the parameter γ is the proportional parameter, and U r and τ m is the reset potential and time constant before the edge device is updated, and Updated reset potential and time constant for this edge device.
[0031] Furthermore, in step A1, first, D edge devices are randomly selected from N edge devices to form the device set of this round as edge devices participating in this round of training; then, the D edge devices in the device set are respectively subjected to local calculation according to steps A11 to A14; N is the total number of edge devices, D is 10% to 30% of N and D is greater than K 1 and K 2 .
[0032] Furthermore, the K 1 and K 2 Both are 2.
[0033] Furthermore, the convolutional spike neural network adopts LIF spike neurons.
[0034] Furthermore, in step A13, the ignition rate Calculated as follows:
[0035]
[0036] in, When the i-th spiking neuron sends a pulse at the t-th time step, it is 1, and when it does not send a pulse, it is 0. I represents the number of spiking neurons in the fully connected layer spiking layer, and T is the preset time step hyperparameter of the convolutional spiking neural network.
[0037] Furthermore, in step A22, the BPTT algorithm is used to perform local training.
[0038] Furthermore, in step A3, the server aggregates the updated local model parameters of each edge device in the received training device set according to the following formula:
[0039]
[0040] in, is the parameter of the global model, M is the training device set, m is the serial number of the edge device in the training device set, and D represents the sum of the number of samples in the local data set of each edge device in the training device set M in the current global training round, D m represents the number of samples in the local dataset of the mth edge device, W m are the parameters of the local model of the mth edge device after local training in this round of global training.
[0041] The beneficial effects of the present invention are:
[0042] (1) In each round of global training, each edge device participating in this round of training uses its own samples to perform T time steps in the local model, and then calculates the firing rate based on the pulse emission at each time step and uploads it to the server. The server selects clients with high firing rates to participate in local training and parameter aggregation updates based on the firing rates uploaded by each edge device participating in this round of training. Since the firing rate can reflect the neuronal activity of the spiking neural network, the more familiar the processed sample is, the lower the firing rate is, and the newer the processed sample is, the higher the firing rate is. Therefore, the server selects edge devices with high firing rates to participate in local training and parameter updates, which can improve the efficiency of sample class balance of heterogeneous devices, thereby improving the convergence speed of federated learning and the accuracy of the model.
[0043] (2) In each round of global training, each edge device participating in this round of training uses its own samples to perform T time steps in the local model, and then calculates the firing rate based on the pulse emission at each time step. At the same time, it also calculates the average membrane potential and uploads it to the server. The average membrane potential can reflect the probability of a neuron being activated, which will ultimately affect the firing rate. Therefore, the server calculates the global average membrane potential of each edge device participating in this round of training, selects the edge device whose average membrane potential deviates greatly from the global average membrane potential, and adjusts the reset potential value and time constant, thereby improving its average membrane potential in the next round of global training, thereby improving its firing rate in the next round. This will gradually balance the firing rates of these devices with those of other devices, further accelerating the efficiency of sample class balance of heterogeneous devices, and further improving the convergence speed of federated learning and the accuracy of the model.
[0044] (3) By making full use of the firing rate characteristics and membrane potential characteristics as mentioned above, the convergence speed of model training is improved overall, so that the required accuracy can be achieved with fewer training rounds, thereby saving power consumption and communication costs.
[0045] The present invention is applicable to the training of image recognition models in vision modules of heterogeneous edge devices such as smart home devices. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 Flow chart of the federated learning method of the edge device vision module in an embodiment of the present invention. DETAILED DESCRIPTION
[0047] The present invention aims to provide a federated learning method for edge device vision modules, and solve the problems of high communication cost, low model accuracy and slow convergence speed in the existing edge device vision modules of smart home edge devices and other edge devices, and the federated learning scheme based on SNN network. The core idea is to make full use of the firing rate characteristics, that is, the firing rate can reflect the neuron activity of the spiking neural network. The more familiar the processed sample is, the lower the firing rate is, and the newer the processed sample is, the higher the firing rate is. Therefore, in each round of global training, each edge device participating in this round of training calculates the firing rate according to the pulse emission situation of each time step after using its own sample to perform T time steps in the local model and uploads it to the server. The server selects the device with high firing rate according to the firing rate of each device to participate in the local training and parameter aggregation update of this round, so that the global model can learn the information of the new sample as soon as possible, thereby improving the efficiency of sample class balance of heterogeneous devices, and then improving the convergence speed of federated learning, and also improving the accuracy of the model, using fewer global training rounds to achieve the required accuracy, saving power consumption and communication costs.
[0048] In addition, the membrane potential can reflect the probability of a neuron being activated, and the probability of a neuron being activated will eventually affect the firing rate. Therefore, as a further optimization solution, this characteristic of the membrane potential is fully utilized. In each round of global training, each edge device participating in this round of training calculates the firing rate and also calculates the average membrane potential and uploads it to the server. The server calculates the global average membrane potential of each edge device participating in this round of training, and selects the edge device whose average membrane potential deviates greatly from the global average membrane potential to adjust the reset potential value and time constant, thereby increasing its average membrane potential in the next round of global training, thereby increasing its firing rate in the next round. This will gradually balance the firing rates of these devices with those of other devices, so as to further accelerate the efficiency of sample class balance of heterogeneous devices, and further improve the convergence speed of federated learning and the accuracy of the model.
[0049] Example
[0050] The federated learning method of the edge device vision module provided in this embodiment is used for training the vision modules in heterogeneous edge devices such as smart phones, smart TVs, and smart computers. The vision module includes a convolutional pulse neural network, which includes a convolutional network and its pulse layer and a fully connected layer and its pulse layer. The purpose is to train a model that can perform image classification tasks on these edge devices.
[0051] Before training, the local sample set and the convolutional spike neural network are first constructed. Specifically, each edge device reads the image data collected by itself, performs data preprocessing and saves it as a Tensor type in the format of Channel×Height×Width, where Channel is the number of channels of the graph, and Height and Width are the shape of the graph. Set a fixed parameter time step T and expand the data format to T×Channel×Height×Width as the local sample set.
[0052] The convolutional spike neural network of this embodiment is constructed based on the LIF (Leaky Integrate-and-Fire) spike neural unit: the output of the convolution layer is connected to the maximum pooling layer, the output of the maximum pooling layer is connected to a spike layer composed of LIF spike neural units, and the output of the spike layer is connected to two excitation layers composed of fully connected layers and LIF spike neural units. The LIF spike neural unit mainly simulates the changes in voltage and current inside the human brain neurons through the three stages of charging, discharging, and resetting, and the output value of the neural unit is only 0 and 1.
[0053] On this basis, the process of federated learning is as follows Figure 1 As shown, the implementation steps include:
[0054] S1. The edge devices participating in this round of training calculate the firing rate and average membrane potential and upload them to the server;
[0055] In this step, for a round of global training, the server randomly selects D edge devices from the N edge devices that need to participate in federated learning to form the device set of this round as the edge devices participating in this round of training; considering the power consumption of the overall system, 0.1-0.3 times the total number of edge devices N is usually taken as the value of D. That is, in each round of global training, 10% to 30% of the edge devices are randomly selected to participate in this round of training.
[0056] At the beginning of training, the server will initialize the parameters of the convolutional pulse neural network and send it to all N edge devices that need to participate in federated learning. Each edge device will save the model locally as a local model. In each subsequent round of training, the D edge devices in the corresponding round of device set will receive the global model updated by the server based on the previous round of training to update the local model.
[0057] In order to utilize the firing rate characteristics and membrane potential characteristics, each edge device in the current round of device set will use the data in its local sample set to test the firing rate and membrane potential, specifically:
[0058] Input the data in the local sample set. After the local model of the edge device completes the calculation of T time steps, calculate its firing rate based on the pulse emission of each spiking neuron in the spiking layer of its fully connected layer at each time step t; calculate the average membrane potential of the edge device based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer.
[0059] In the specific neuron calculation process, the input pulse data of the lth layer neuron is I (l) (t), according to the charging, discharging and potential reset formulas of LIF itself, the input pulse data I of the l+1th layer is calculated. (l+1) (t).
[0060] Assuming there are M layers of neurons, for the last layer M, we need to record its membrane potential V M and firing rate The membrane potential at time t is calculated by the following formula:
[0061] V (M) (t) = f(U (M) (t-1),I (M) (t))
[0062] Among them, V (M) (t) represents the membrane potential of the Mth layer at time t, and the f function can be further expanded as the following formula.
[0063]
[0064] Among them, U (M) (t-1) is the initial potential of the neuron at time t-1, I (M) (t) is the input current of the Mth layer at time t. r is the reset potential value, τ m is the time constant. Both parameters are hyperparameters during the experiment. The value of the reset potential affects the sensitivity of the neuron, that is, the degree of response of the neuron to the input.
[0065] If the membrane potential reaches the activation threshold, a discharge operation is performed to generate a pulse value of 1. The formula is as follows:
[0066] S (l) (t) = g(V (l) (t)-θ)
[0067] S (l) (t) represents the pulse value generated by the discharge of the lth layer of neurons at time t, θ is the activation threshold, and the function g(x) can be further described as the following formula:
[0068]
[0069] Calculate the membrane potential V at all times within the time step T (M) (t), the mean is calculated as the average membrane potential V M And record.
[0070] use It means that the i-th neuron sends a pulse of 1 at the t-th time step. Then the firing rate of all I neurons in the last layer within T time steps can be calculated by the following formula:
[0071]
[0072] in, When the i-th spiking neuron sends a pulse at the t-th time step, it is 1, and when it does not send a pulse, it is 0. I represents the number of spiking neurons in the fully connected layer spiking layer, and T is the preset time step hyperparameter of the convolutional spiking neural network.
[0073] Finally, each of the D edge devices will calculate the firing rate and the average membrane potential V M Upload server.
[0074] S2. The server updates the membrane potential parameters of the device according to the average membrane potential screening device, performs local training according to the firing rate screening device, and uploads the updated model parameters to the server;
[0075] In this step, the server receives the firing rate data uploaded by D edge devices. and the average membrane potential V M After that, the ignition rate and the average membrane potential V M Sort in descending order, forming a sorted list f list and V list .
[0076] Next, from the sorted list f list Select the top K with the highest ignition rate 1 The edge devices are used as the devices to be trained locally, forming the training device set for this round. list Take the average of all the average membrane potentials in and get the global average membrane potential Then from the sorted list V list The top K 2 An edge device is used as the device for which membrane potential parameters are to be updated.
[0077] In an exemplary solution, in order to balance the computing power consumption and accuracy, K 1 and K 2 The value is 2. For the convenience of description, the two devices selected for local training are recorded as client x With client y , the two devices selected for membrane potential parameter update are recorded as client a With client b .
[0078] 1. Client a With client b The membrane potential parameters are updated:
[0079] The membrane potential parameters that need to be updated include the reset potential and the time constant. The server first needs to calculate the client a With client b The difference between the average membrane potential and the global membrane potential is then used to calculate the updated reset potential and time constant, respectively, according to the following formulas:
[0080]
[0081]
[0082] Among them, V k2 is the average membrane potential of the k2th edge device, x k2 is the difference between the average membrane potential of the k2th edge device and the global average membrane potential, the parameter γ is the proportional parameter, and U r and τ mis the reset potential and time constant before the edge device is updated, and Updated reset potential and time constant for this edge device.
[0083] The server will calculate the client a With client b The updated reset potential and time constant are sent to the client accordingly. a With client b ,client a With client b The reset potential and time constant in the local model are updated according to the updated reset potential and time constant, so as to improve its average membrane potential in the next round of global training, thereby improving its firing rate in the next round of global training.
[0084] 2. Client x With client y Local training:
[0085] For client x With client y , respectively using its local data set and local model, and performing local training according to a preset number of local training rounds (such as 5 to 15 rounds) to obtain its updated local model parameters.
[0086] In an exemplary solution, local training can use the BPTT algorithm. The full name of BPTT is Backpropagation Through Time. When the data in the local data set is input and the model gives a prediction result, the error is calculated according to the following formula:
[0087]
[0088] Among them, j represents the predicted label and y is the true value label.
[0089] against and Where t represents the tth time step and l represents the lth layer of the network. represents the pulse value generated by the neuron for the predicted label, Represents the impulse value generated for the true label. After obtaining the error L, the gradient update of the error relative to the weight can be further calculated using the chain rule, as follows:
[0090]
[0091] Among them, S is the spike value generated by the pulse neuron, which can only have a value of 0 or 1. U is the membrane potential value of the neuron.
[0092] At the same time, considering There may be a gradient vanishing problem, that is, when S is 0, further chain derivation cannot be performed. The proxy gradient method is used to further calculate the gradient according to the pulse value of the neuron. The formula is as follows:
[0093]
[0094] Among them, θ is the activation threshold of the spiking neuron. When a pulse is generated, the derivative value of this part is set to 1, otherwise it is set to 0.
[0095] According to the above training method, client x With client y After local training is completed, the updated local model parameters will be uploaded to the server.
[0096] It should be noted that in actual scenarios, the devices selected for local training and the devices for membrane potential parameter update may overlap, for example: client x With client a or client b Is the same device, or client y With client a or client b In this case, the membrane potential parameters of the same device can be updated first, and then local training can be performed.
[0097] S3. The server aggregates the updated model parameters uploaded by the locally trained devices and updates the global model.
[0098] In this step, after receiving the updated model parameters uploaded by the locally trained devices selected in the current global training round, the server performs parameter aggregation according to the following formula:
[0099]
[0100] in, is the parameter of the global model, M is the training device set, m is the serial number of the edge device in the training device set, and D represents the sum of the number of samples in the local data set of each edge device in the training device set M in the current global training round, D m represents the number of samples in the local dataset of the mth edge device, W m are the parameters of the local model of the mth edge device after local training in this round of global training.
[0101] After the server aggregates according to the above formula and obtains the updated model parameters, it updates the local global model and sends the updated model parameters to all N edge devices participating in federated learning.
[0102] S4, repeat steps S1-S3 to perform iterative training;
[0103] In this step, first determine whether the preset training completion conditions are met, such as reaching the set global training rounds or the accuracy of the model meets the requirements. If so, stop training. The global model at this time is the convolutional spiking neural network that has completed training. Send the final model parameters to all N edge devices participating in federated learning, and each edge device will obtain the trained convolutional spiking neural network to perform image recognition tasks. Otherwise, return to step S1 for the next round of global training.
[0104] Experimental verification:
[0105] In order to verify the accuracy and convergence speed of the federated learning scheme provided by the present invention, three common federated learning schemes in the prior art, Rand, AFL, and FedProx, are verified with this scheme on three different data sets, MNIST, FMNIST, and CIFAR-10, respectively. The results are shown in Tables 1 and 2, where Table 1 shows the accuracy performance on different data sets, and Table 2 shows the communication round performance to achieve the specified accuracy. α in the two tables indicates the degree of data heterogeneity between clients. The larger the number, the more uniform the distribution of values between clients.
[0106] Table 1: Accuracy of different solutions on different datasets
[0107]
[0108] Table 2: Communication rounds for different schemes to achieve a specified accuracy on different datasets
[0109]
[0110] It can be seen from Table 1 that the scheme (Ours in the table) has the highest accuracy on three different data sets compared with the above three schemes in the prior art. And it can be seen from Table 2 that the scheme (Ours in the table) can use the least communication rounds on three different data sets to make the model reach the specified accuracy, thereby accelerating the convergence efficiency.
[0111] Finally, it should be noted that the above embodiments are only preferred implementations and are not intended to limit the present invention. It should be pointed out that for those skilled in the art, several modifications, equivalent replacements, improvements, etc. can be made without departing from the scope of the present invention and the scope of protection of the claims, and all of these should be included in the protection scope of the present invention.
Claims
1. A federated learning method for edge device vision module, characterized in that , The edge device vision module includes a convolutional spiking neural network, which includes a convolutional network and its spiking layer and a fully connected layer and its spiking layer; the training of the convolutional spiking neural network includes the following steps: A1. Each edge device participating in this round of training performs local calculations according to the following steps: A11. The edge device receives the global model sent by the server and updates its local model. A12. Extract a local sample from the local sample set of the edge device and input the sample into the local model of the edge device. A13. After the local model of the edge device completes the calculation of T time steps, the firing rate is calculated based on the pulse emission of each spiking neuron in the spiking layer of its fully connected layer at each time step t; T is a preset hyperparameter representing the time step of the convolutional spiking neural network; A14, the edge device uploads its ignition rate to the server; A2. Select edge devices and perform local training according to the following steps: A21. Select the first K1 edge devices with the largest firing rate from each edge device to form the training device set of this round, where K1 is an integer ≥ 2; A22. For each edge device in the training device set, use its local data set and local model to perform local training according to a preset number of local training rounds to obtain its updated local model parameters; A23. Each edge device in the training device set uploads its updated local model parameters to the server respectively. A3. The server aggregates the updated local model parameters of each edge device in the received training device set, and updates the global model based on the aggregation results; A4. Determine whether the preset completion condition is met. If so, stop training. The global model at this time is the convolutional spike neural network that has completed training. Otherwise, return to step A1 to perform the next round of global training.
2. A federated learning method for an edge device vision module as described in claim 1, characterized in that , Step A13 further includes: after the local model of the edge device completes the calculation of T time steps, calculating the average membrane potential of the edge device based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer; In step A14, the edge device uploads its firing rate and average membrane potential to the server; Before step A21 of step A2, it also includes: based on the average membrane potential of each edge device, performing mean calculation to obtain the global average membrane potential of this round of training; based on the global average membrane potential, screening and obtaining the top K2 edge devices with the largest difference therewith; for the K2 edge devices obtained by the screening, using the difference between their average membrane potential and the global average membrane potential, respectively, updating their reset potential and time constant; the K2 is an integer ≥2.
3. A federated learning method for an edge device vision module as described in claim 2, characterized in that , Update its reset potential and time constant according to the following formula: Among them, V k2 is the average membrane potential of the k2th edge device, x k2 is the difference between the average membrane potential of the k2th edge device and the global average membrane potential, the parameter γ is the proportional parameter, and U r and τ m is the reset potential and time constant before the edge device is updated, and Updated reset potential and time constant for this edge device.
4. A federated learning method for an edge device vision module as described in claim 2, characterized in that , In step A1, first, D edge devices are randomly selected from N edge devices to form the device set of this round, as the edge devices participating in this round of training; then, the D edge devices in the device set are respectively subjected to local calculations according to steps A11 to A14; N is the total number of edge devices, D is 10% to 30% of N, and D is greater than K1 and K2.
5. A federated learning method for an edge device vision module as described in claim 4, characterized in that , The K1 and K2 are both 2.
6. A method for federated learning of an edge device vision module as described in any one of claims 1 to 5, characterized in that ,The convolutional spike neural network adopts LIF spike neurons.
7. A federated learning method for an edge device vision module as described in any one of claims 1 to 5, characterized in that In step A13, the ignition rate Calculated as follows: in, When the i-th spiking neuron sends a pulse at the t-th time step, it is 1, and when it does not send a pulse, it is 0. I represents the number of spiking neurons in the fully connected layer spiking layer, and T is the preset time step hyperparameter of the convolutional spiking neural network.
8. A federated learning method for an edge device vision module as described in any one of claims 1 to 5, characterized in that ,In step A22, BPTT algorithm is used to perform local training.
9. A method for federated learning of an edge device vision module as described in any one of claims 1 to 5, characterized in that ,In step A3, the server aggregates the updated local model parameters of each edge device in the received training device set according to the following formula: in, is the parameter of the global model, M is the training device set, m is the serial number of the edge device in the training device set, and D represents the sum of the number of samples in the local data set of each edge device in the training device set M in the current global training round, D m represents the number of samples in the local dataset of the mth edge device, W m are the parameters of the local model of the mth edge device after local training in this round of global training.
Citation Information
Patent Citations
Federal learning method for dynamic weight in edge scene
CN115526333A
Edge intelligence method for hybrid distributed learning and centralized learning
CN115936109A
Federal learning optimization method and system based on reinforcement learning
CN116523079A
Federal learning method of pulse neural network for flaw detection
CN117875408A
Federal learning method and device for adaptive communication in dynamic bandwidth scene
CN117938690A
