A federated learning method of edge device vision module

Abstract

The present application relates to a federal learning technology, which discloses a federal learning method of an edge device visual module, and solves the problems of high communication cost, low model accuracy and slow convergence speed existing in the federal learning scheme of the existing edge device visual module based on SNN network. In the present application, in one global training round, the edge device participating in the training calculates the firing rate and uploads it to the server; the server filters the devices for local training according to the firing rate and uploads the updated model parameters to the server; the server aggregates the updated model parameters uploaded by the devices for local training, updates the global model, and distributes the updated global model to each edge device. Through the above steps, iterative training is carried out, and finally the trained model is obtained. The present application is suitable for training of image recognition model in heterogeneous edge device visual modules such as smart home devices.

Description

Technical Field

[0001] This invention relates to federated learning technology, and more specifically to a federated learning method for a vision module of an edge device. Background Technology

[0002] Edge device vision modules capture, process, and analyze images to identify and capture objects, supporting subsequent machine vision-based decision-making. With the development of IoT technology, the application of edge device vision modules is becoming increasingly widespread, such as in smart home devices. As a result, the issue of privacy breaches is becoming increasingly prominent and important. Privacy protection for edge device vision modules faces two main challenges: firstly, users do not want their privacy data to be leaked and demand the right to know and control over how their data is used; secondly, device manufacturers want to collect as much user data as possible to provide better intelligent services.

[0003] Therefore, effectively protecting user privacy data in the vision modules of edge devices has become a critical issue that urgently needs to be addressed. Federated learning can effectively solve this data privacy problem. Federated learning is a variant of distributed learning that aims to protect data privacy while allowing multiple edge devices to collaboratively train a model, thereby giving the model better adaptability. The main process of federated learning is as follows: the server initializes the global model and sends it to the edge devices participating in the federated learning. Each edge device trains the model locally using its own stored data and transmits the training parameters to the server. The server aggregates the parameters uploaded by each edge device, updates the parameters of the global model, and redistributes it to the edge devices. Each edge device then trains the new global model locally, and this process is repeated until a fully trained global model is obtained and distributed to all the edge devices participating in the federated learning.

[0004] Edge devices are diverse, exhibiting varying communication capabilities, stability, and local computing power, resulting in a complex and heterogeneous structure. During federated training, edge devices may experience data loss or communication interruptions due to their inherent limitations, impacting the overall performance of the federated learning system. Therefore, addressing the variable terminal problem in federated learning systems is crucial for the application and deployment of vision modules on edge devices, such as in smart home IoT systems.

[0005] Currently, research on federated learning systems with variable terminal devices includes studies on both Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs), as detailed below:

[0006] (1) In the case of ANN, techniques such as asynchronous federated learning and knowledge distillation are commonly used to address the variable terminal device problem in federated learning. Asynchronous federated learning adapts to the uncertainty of variable terminal devices by allowing terminal devices to complete local model updates at different times. Distillation learning addresses the problem of insufficient effective information received by terminal devices due to variable terminal devices by transferring knowledge from the global model to the local model, thereby enabling the local model to learn better.

[0007] (2) In the SNN field, by combining more computationally sparsity and simplicity spiking neural units with federated learning, a federated learning system with lower power consumption and computational requirements than the ANN model method has been realized, making it adaptable to a wider variety of terminal devices. These methods improve the model training and model aggregation methods in the federated learning system to adapt it to spiking neural networks, and also study the accuracy and robustness of SNN networks in the federated learning system.

[0008] While asynchronous federated learning methods using ANNs can mitigate the impact of terminal variability to some extent, the model parameters updated by each device can have unstable effects on other devices, leading to a relatively slow overall convergence speed and making optimization and convergence and model performance analysis difficult. Knowledge distillation methods for ANNs suffer from information loss because the knowledge of the global model is compressed and passed to local models, potentially failing to fully represent the complexity of the global model. Furthermore, improper knowledge distillation can cause local models to over-rely on specific features of the global model, resulting in severe overfitting.

[0009] While current research on the combination of SNN and federated learning shows that this method can achieve better power consumption performance than ANN, enabling SNN to cope with more complex and resource-constrained edge devices, SNN models under federated learning systems still suffer from problems such as high communication costs, low model accuracy, and slow convergence speed. Summary of the Invention

[0010] The technical problem to be solved by this invention is to propose a federated learning method for edge device vision modules, which solves the problems of high communication cost, low model accuracy and slow convergence speed of federated learning schemes based on SNN networks for edge device vision modules such as smart home edge devices.

[0011] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0012] A federated learning method for an edge device vision module, the edge device vision module including a convolutional spiking neural network, the convolutional spiking neural network including a convolutional network and its spiking layers and fully connected layers and their spiking layers; the training of the convolutional spiking neural network includes the following steps:

[0013] A1. Each edge device participating in this round of training shall perform local computation according to the following steps:

[0014] A11. Edge devices receive the global model from the server and update their local models.

[0015] A12. Extract a local sample from the local sample set of the edge device and input it into the local model of the edge device;

[0016] A13. After the local model of the edge device completes the calculation of T time steps, the firing rate is calculated based on the pulse emission of each spiking neuron in the spiking layer of its fully connected layer at each time step t; where T is a preset hyperparameter, representing the time step of the convolutional spiking neural network.

[0017] A14. The edge device uploads its ignition rate to the server;

[0018] A2. Follow these steps to select edge devices and perform local training:

[0019] A21. From each edge device, select the top K1 edge devices with the highest ignition rate to form the training device set for this round, where K1 is an integer ≥2;

[0020] A22. For each edge device in the training device set, use its local dataset and local model to perform local training according to the preset number of local training rounds to obtain its updated local model parameters.

[0021] A23. Each edge device in the training device set uploads its updated local model parameters to the server.

[0022] A3. The server aggregates the updated local model parameters of each edge device in the received training device set, and updates the global model based on the aggregation results.

[0023] A4. Determine whether the preset completion conditions have been met. If yes, stop training. The global model at this point is the completed convolutional spiking neural network. Otherwise, return to step A1 and proceed to the next round of global training.

[0024] Furthermore, step A13 also includes: after the local model of the edge device completes the calculation of T time steps, the average membrane potential of the edge device is calculated based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer.

[0025] In step A14, the edge device uploads its ignition rate and average membrane potential to the server;

[0026] Before step A21 in step A2, the method further includes: calculating the mean value based on the average membrane potential of each edge device to obtain the global average membrane potential for this round of training; selecting the top K2 edge devices with the largest difference from the global average membrane potential based on the global average membrane potential; updating the reset potential and time constant of the selected K2 edge devices using the difference between their average membrane potential and the global average membrane potential; where K2 is an integer ≥2.

[0027] Furthermore, update its reset potential and time constant according to the following formula:

[0028]

[0029]

[0030] Among them, V k2 Let x be the average membrane potential of the k2th edge device. k2 U is the difference between the average membrane potential of the k2th edge device and the global average membrane potential, where γ is the proportionality parameter. r and τ m This refers to the reset potential and time constant of the edge device before the update. and This refers to the updated reset potential and time constant for the edge device.

[0031] Furthermore, in step A1, firstly, D edge devices are randomly selected from N edge devices to form the device set for this round, which will be used as edge devices to participate in this round of training; then, the D edge devices in the device set are used for local calculations according to steps A11 to A14 respectively; where N is the total number of edge devices, D is 10% to 30% of N and D is greater than K1 and K2.

[0032] Furthermore, both K1 and K2 are 2.

[0033] Furthermore, the convolutional spiking neural network employs LIF spiking neurons.

[0034] Furthermore, in step A13, the ignition rate Calculate using the following formula:

[0035]

[0036] in, At time step t, the ith spiking neuron emits a pulse (1) and does not emit a pulse (0). I represents the number of spiking neurons in the fully connected spiking layer, and T is the preset time step hyperparameter of the convolutional spiking neural network.

[0037] Furthermore, in step A22, the BPTT algorithm is used for local training.

[0038] Furthermore, in step A3, the server aggregates the updated local model parameters of each edge device in the received training device set according to the following formula:

[0039]

[0040] in, Here, M represents the parameters of the global model, m represents the training device set, m represents the index of the edge device in the training device set, and D represents the sum of the number of samples in the local datasets of each edge device in the training device set M during the current global training round. m W represents the number of samples in the local dataset of the m-th edge device. m These are the parameters of the local model for the m-th edge device after local training in this round of global training.

[0041] The beneficial effects of this invention are:

[0042] (1) In each round of global training, each edge device participating in this round of training calculates the ignition rate of the local model after using its own samples for T time steps. The server then uploads the ignition rate to the server based on the pulse emission of each time step. The server selects clients with high ignition rates to participate in local training and parameter aggregation and update based on the ignition rates uploaded by each edge device participating in this round of training. Since the ignition rate can reflect the activity of neurons in the spiking neural network, the more familiar the processed samples are, the lower the ignition rate will be, while the newer the processed samples are, the higher the ignition rate will be. Therefore, the server selects edge devices with high ignition rates to participate in local training and parameter update, which can improve the efficiency of inter-class balance of samples of heterogeneous devices, thereby improving the convergence speed of federated learning and also improving the accuracy of the model.

[0043] (2) In each round of global training, each edge device participating in this round of training calculates the firing rate based on the pulse emission of each time step in the local model using its own samples, and also calculates the average membrane potential and uploads it to the server. The average membrane potential can reflect the probability of neuron activation, and the probability of neuron activation will eventually affect the firing rate. Therefore, the server calculates the global average membrane potential of each edge device participating in this round of training, selects the edge devices with large deviations from the global average membrane potential, and adjusts the reset potential value and time constant to improve their average membrane potential in the next round of global training, thereby improving their firing rate in the next round. This will make the firing rate of these devices gradually tend to be balanced with the firing rate of other devices, so as to further accelerate the inter-class balance efficiency of heterogeneous devices and further improve the convergence speed and accuracy of federated learning.

[0044] (3) By making full use of the ignition rate characteristics and membrane potential characteristics as described above, the convergence speed of model training is improved as a whole, so that the required accuracy can be achieved with fewer training rounds, thereby saving power consumption and communication costs.

[0045] This invention is applicable to the training of image recognition models in the vision modules of heterogeneous edge devices such as smart home devices. Attached Figure Description

[0046] Figure 1 This is a flowchart of the federated learning method for the edge device vision module in an embodiment of the present invention. Detailed Implementation

[0047] This invention aims to provide a federated learning method for edge device vision modules, addressing the problems of high communication costs, low model accuracy, and slow convergence speed in existing federated learning schemes based on SNN networks for edge device vision modules, such as those used in smart home edge devices. The core idea is to fully utilize the ignition rate characteristic; that is, the ignition rate reflects the activity of neurons in a spiking neural network. The more familiar the processed samples, the lower the ignition rate, while the newer the processed samples, the higher the ignition rate. Therefore, in each round of global training, each edge device participating in this round performs T time-step calculations on its local model using its own samples. Based on the pulse emission at each time step, the device calculates its ignition rate and uploads it to the server. The server then selects devices with high ignition rates to participate in the local training and parameter aggregation update for this round. This allows the global model to learn information from new samples as quickly as possible, improving the efficiency of inter-class balance among heterogeneous devices, thereby increasing the convergence speed of federated learning and improving model accuracy. The required accuracy can be achieved with fewer global training rounds, saving power consumption and communication costs.

[0048] Furthermore, membrane potential reflects the probability of neuron activation, which ultimately affects the firing rate. Therefore, as a further optimization strategy, this characteristic of membrane potential is fully utilized. In each round of global training, each edge device participating in the current round calculates its firing rate and also uploads its average membrane potential to the server. The server calculates the global average membrane potential of each edge device participating in the current round of training, selects edge devices with average membrane potentials that deviate significantly from the global average membrane potential, and adjusts their reset potential values ​​and time constants to improve their average membrane potential in the next round of global training, thereby improving their firing rate in the next round. This will cause the firing rates of these devices to gradually approach equilibrium with the firing rates of other devices, further accelerating the inter-class balance efficiency of heterogeneous devices, and thus further improving the convergence speed and model accuracy of federated learning.

[0049] Example

[0050] The federated learning method for edge device vision modules provided in this embodiment is used for training vision modules in heterogeneous edge devices such as smartphones, smart TVs, and smart computers. The vision module includes a convolutional spiking neural network, which includes a convolutional network and its spiking layer and a fully connected layer and its spiking layer. The purpose is to train a model that can perform image classification tasks on these edge devices.

[0051] Before training, a local sample set and a convolutional spiking neural network are constructed. Specifically, each edge device reads its own acquired image data, performs data preprocessing, and saves it as a Tensor in the format Channel×Height×Width, where Channel represents the number of channels in the image, and Height and Width represent the shape of the image. A fixed time step T is set, and the data format is expanded to T×Channel×Height×Width to serve as the local sample set.

[0052] The convolutional spiking neural network in this embodiment is constructed based on LIF (Leaky Integrate-and-Fire) spiking neural units: the output of the convolutional layer is connected to a max pooling layer, the output of the max pooling layer is connected to a spiking layer composed of LIF spiking neural units, and the output of the spiking layer is connected to two activation layers composed of fully connected layers and LIF spiking neural units. The LIF spiking neural unit mainly simulates the changes in voltage and current inside human brain neurons through three stages: charging, discharging, and resetting, and the output value of this neural unit is only 0 and 1.

[0053] Based on this, the process of federated learning is as follows: Figure 1 As shown, the implementation steps include the following:

[0054] S1. Edge devices participating in this round of training calculate the ignition rate and average membrane potential and upload them to the server.

[0055] In this step, for each round of global training, the server randomly selects D edge devices from the N edge devices that need to participate in federated learning, forming the device set for this round, which will be used as the edge devices participating in this round of training. Considering the overall system power consumption, the value of D is usually taken as 0.1-0.3 times the total number of edge devices N. That is, in each round of global training, 10% to 30% of the edge devices are randomly selected to participate in this round of training.

[0056] At the beginning of training, the server initializes the parameters of the convolutional spiking neural network and sends it to all N edge devices that need to participate in federated learning. Each edge device saves the model locally as its local model. In each subsequent training round, the D edge devices in the corresponding round's device set receive the global model updated by the server based on the previous training round and update their local models accordingly.

[0057] To utilize ignition rate and membrane potential characteristics, each edge device in the current round of the device set will use data from its own local sample set to test ignition rate and membrane potential, specifically:

[0058] Input data from the local sample set. After the local model of the edge device completes the calculation for T time steps, calculate the firing rate of each spiking neuron in the spiking layer of its fully connected layer at each time step t. Calculate the average membrane potential of the edge device based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer.

[0059] In the specific neuron computation process, the input pulse data of the l-th layer neuron is I. (l) (t), based on the LIF's own charging, discharging, and potential reset formulas, the input pulse data I of the (l+1)th layer is calculated. (l+1) (t).

[0060] Suppose there are M layers of neurons. For the last layer M, we need to record its membrane potential V. M and ignition rate The membrane potential at time t is calculated using the following formula:

[0061] V (M) (t)=f(U (M) (t-1),I (M) (t))

[0062] Among them, V (M) (t) represents the membrane potential of the Mth layer at time t. The f function can be further extended and written as the following formula.

[0063]

[0064] Among them, U (M) (t-1) represents the initial potential of the neuron at time t-1, I (M) (t) represents the input current of the M-th layer at time t. r τ is the reset potential value. m The time constant is one of the two parameters, which are hyperparameters used in the experiment. The value of the reset potential affects the sensitivity of the neuron, that is, the degree of response of the neuron to the input.

[0065] If the membrane potential reaches the activation threshold, a discharge operation is performed, generating a pulse value of 1, as described by the following formula:

[0066] S (l) (t)=g(V (l) (t)-θ)

[0067] S (l) (t) represents the pulse value generated by the firing of the l-th layer neuron at time t, θ is the activation threshold, and the function g(x) can be further described by the following formula:

[0068]

[0069] Calculate the membrane potential V at all times within time step T. (M) After (t), the mean value is calculated and used as the average membrane potential V. M And record it.

[0070] use If the i-th neuron emits pulse 1 at time step t, then the firing rate of all I neurons in the last layer over T time steps can be calculated using the following formula:

[0071]

[0072] in, At time step t, the ith spiking neuron emits a pulse (1) and does not emit a pulse (0). I represents the number of spiking neurons in the fully connected spiking layer, and T is the preset time step hyperparameter of the convolutional spiking neural network.

[0073] Finally, each of the D edge devices will submit its calculated ignition rate. and average membrane potential V M Upload to the server.

[0074] S2. The server updates the membrane potential parameters of the screening device based on the average membrane potential, and performs local training on the screening device based on the ignition rate and uploads the updated model parameters to the server.

[0075] In this step, the server receives the ignition rates uploaded by D edge devices. and average membrane potential V M Then, the ignition rate was assessed separately. and average membrane potential V M Sort in descending order to form a sorted list f. list and V list .

[0076] Next, from the sorted list f list The top K1 edge devices with the highest ignition rates are selected as the devices to be trained locally, forming the training device set for this round. This is then processed by sorting the list V... list The average of all average membrane potentials is calculated to obtain the global average membrane potential. Then from sorted list V list The top K2 edge devices with the highest average membrane potential were selected as the devices for which membrane potential parameters need to be updated.

[0077] In one exemplary scheme, considering the balance between computational power consumption and accuracy, both K1 and K2 are set to 2. For ease of description, the two devices selected for local training are denoted as clients. x With client y The two devices selected for membrane potential parameter updates are denoted as clients. a With client b .

[0078] I. Client a With client b Membrane potential parameters updated:

[0079] The membrane potential parameters that need to be updated include the reset potential and the time constant. The server first needs to calculate these separately for the client. a With client b The difference between the average membrane potential and the global membrane potential is used to calculate the updated reset potential and time constant, respectively, according to the following formulas:

[0080]

[0081]

[0082] Among them, V k2 Let x be the average membrane potential of the k2th edge device. k2 U is the difference between the average membrane potential of the k2th edge device and the global average membrane potential, where γ is the proportionality parameter. r and τ m This refers to the reset potential and time constant of the edge device before the update. and This refers to the updated reset potential and time constant for the edge device.

[0083] The server will calculate the client based on the above formula. a With client b The updated reset potential and time constant are sent to the client accordingly. a With client b client a With client b The reset potential and time constant in the local model are updated based on the updated reset potential and time constant, thereby improving its average membrane potential in the next round of global training and thus improving its ignition rate in the next round of global training.

[0084] II. client x With client y Local training:

[0085] For client x With client y Using its local dataset and local model respectively, local training is performed according to a preset number of local training rounds (such as 5 to 15 rounds) to obtain its updated local model parameters.

[0086] In one exemplary approach, local training can employ the BPTT algorithm, which stands for Backpropagation Through Time. When input data from the local dataset, after the model provides its predictions, the error is calculated using the following formula:

[0087]

[0088] Where j represents the predicted label and y represents the true label.

[0089] against and Where t represents the t-th time step, and l represents the l-th layer of the network. This represents the pulse value generated by the neuron in response to the predicted label. This represents the impulse value generated for the true label. After obtaining the error L, the gradient update of the error relative to the weights can be further calculated using the chain rule, as shown in the following formula:

[0090]

[0091] Where S is the peak value generated by the spiking neuron, which has only a value of 0 or 1. U is the membrane potential value of the neuron.

[0092] At the same time, considering The potential gradient vanishing problem, where further chain rule differentiation is impossible when S is 0, necessitates the use of a surrogate gradient to perform further gradient calculations based on the impulse values ​​generated by the neuron. The formula is as follows:

[0093]

[0094] Here, θ is the activation threshold of the spiking neuron. When a spiking neuron is generated, the derivative value of this part is set to 1, otherwise it is set to 0.

[0095] Following the training method described above, the client x With client y After local training is complete, the updated local model parameters will be uploaded to the server.

[0096] It should be noted that in real-world scenarios, there may be overlap between the devices selected for local training and the devices selected for membrane potential parameter updates, for example: client x With client a or client b Is it the same device, or the client? y With client a or client b This applies to the same device. In this case, the membrane potential parameters can be updated first for the same device, and then local training can be performed.

[0097] S3. The server aggregates the updated model parameters uploaded by the locally trained devices and updates the global model.

[0098] In this step, after receiving the updated model parameters uploaded by locally trained devices selected from the current global training round, the server aggregates the parameters according to the following formula:

[0099]

[0100] in, Here, M represents the parameters of the global model, m represents the training device set, m represents the index of the edge device in the training device set, and D represents the sum of the number of samples in the local datasets of each edge device in the training device set M during the current global training round. m W represents the number of samples in the local dataset of the m-th edge device. m These are the parameters of the local model for the m-th edge device after local training in this round of global training.

[0101] After the server aggregates the data according to the above formula and obtains the updated model parameters, it updates the local global model and distributes the updated model parameters to all N edge devices participating in federated learning.

[0102] S4. Repeat steps S1-S3 for iterative training;

[0103] In this step, it is first determined whether the preset training completion conditions have been met, such as reaching the set number of global training rounds or the model's accuracy meeting the requirements. If so, training stops, and the global model at this point is the completed convolutional spiking neural network. The final model parameters are then distributed to all N edge devices participating in federated learning, so each edge device will receive the trained convolutional spiking neural network to perform the image recognition task. Otherwise, the process returns to step S1 to proceed to the next round of global training.

[0104] Experimental verification:

[0105] To verify the accuracy and convergence speed of the federated learning scheme provided in this invention, three common existing federated learning schemes—Rand, AFL, and FedProx—were compared with this scheme on three different datasets: MNIST, FMNIST, and CIFAR-10. The results are shown in Tables 1 and 2, where Table 1 shows the accuracy performance on different datasets, and Table 2 shows the performance in the number of communication rounds required to achieve the specified accuracy. In both tables, α represents the degree of data heterogeneity among clients; a larger number indicates a more uniform numerical distribution among clients.

[0106] Table 1: Accuracy of different schemes on different datasets

[0107]

[0108] Table 2: Number of communication rounds required for different schemes to achieve the specified accuracy on different datasets

[0109]

[0110] As can be seen from Table 1, our proposed solution (Ours in the table) has the highest accuracy on all three different datasets compared to the three existing solutions. As can be seen from Table 2, our proposed solution (Ours in the table) can achieve the specified accuracy with the fewest communication rounds on all three different datasets, thereby accelerating the convergence efficiency.

[0111] Finally, it should be noted that the above embodiments are merely preferred embodiments and are not intended to limit the present invention. It should be pointed out that those skilled in the art can make various modifications, equivalent substitutions, and improvements without departing from the spirit and scope of the claims, and all such modifications, substitutions, and improvements should be included within the scope of protection of the present invention.

Claims

1. A federated learning method for a vision module of an edge device, characterized in that... , The edge device vision module includes a convolutional spiking neural network, which comprises a convolutional network and its spiking layers, and a fully connected layer and its spiking layers; the training of the convolutional spiking neural network includes the following steps: A1. Each edge device participating in this round of training shall perform local computation according to the following steps: A11. Edge devices receive the global model from the server and update their local models. A12. Extract a local sample from the local sample set of the edge device and input it into the local model of the edge device; A13. Complete the local model on the edge device. After calculation at each time step, the spiking neurons of the spiking layer based on its fully connected layer at each time step The ignition rate is calculated based on the pulse emission situation; These are preset hyperparameters, representing the time steps of the convolutional spiking neural network; A14. The edge device uploads its ignition rate to the server; A2. Follow these steps to select edge devices and perform local training: A21. From all peripheral devices, select the one with the highest ignition rate. These edge devices constitute the training device set for this round. for Integers; A22. For each edge device in the training device set, use its local dataset and local model to perform local training according to the preset number of local training rounds to obtain its updated local model parameters. A23. Each edge device in the training device set uploads its updated local model parameters to the server. A3. The server aggregates the updated local model parameters of each edge device in the received training device set, and updates the global model based on the aggregation results. A4. Determine whether the preset completion conditions have been met. If so, stop training. At this point, the global model is the completed convolutional spiking neural network. Otherwise, return to step A1 and proceed to the next round of global training; Step A13 also includes: completing the local model on the edge device. After calculating each time step, the average membrane potential of the edge device is calculated based on the membrane potential of each spiking neuron in the spiking layer of its fully connected layer. In step A14, the edge device uploads its ignition rate and average membrane potential to the server; Before step A21 in step A2, the method further includes: calculating the mean value based on the average membrane potential of each edge device to obtain the global average membrane potential for this round of training; and selecting the top edge devices with the largest difference from the global average membrane potential based on the global average membrane potential. One edge device; for the filtered results Each edge device updates its reset potential and time constant using the difference between its average membrane potential and the global average membrane potential; for Integers.

2. The federated learning method for an edge device vision module as described in claim 1, characterized in that... , Update its reset potential and time constant using the following formula: in, For the first Average membrane potential of each edge device For the first The difference between the average membrane potential of each edge device and the global average membrane potential, parameters For proportional parameters, and This refers to the reset potential and time constant of the edge device before the update. and This refers to the updated reset potential and time constant for the edge device.

3. The federated learning method for an edge device vision module as described in claim 1, characterized in that... , In step A1, firstly, from Randomly select from the edge devices Several edge devices constitute the device set for this round, serving as the edge devices participating in this round of training; then, the devices in the device set... Each edge device performs local computation according to steps A11-A14; This represents the total number of edge devices. for 10%~30% and Greater than and .

4. The federated learning method for an edge device vision module as described in claim 3, characterized in that... , The and Both are 2.

5. A federated learning method for an edge device vision module as described in any one of claims 1 to 4, characterized in that... The convolutional spiking neural network uses LIF spiking neurons.

6. A federated learning method for an edge device vision module as described in any one of claims 1 to 4, characterized in that... In step A13, the ignition rate Calculate using the following formula: in, In the The spiking neuron in the first... A pulse is 1 if it is emitted at each time step, and 0 if no pulse is emitted. This indicates the number of spiking neurons in the fully connected spiking layer. These are the preset time step hyperparameters for the convolutional spiking neural network.

7. A federated learning method for an edge device vision module as described in any one of claims 1 to 4, characterized in that... In step A22, the BPTT algorithm is used for local training.

8. A federated learning method for an edge device vision module as described in any one of claims 1 to 4, characterized in that... In step A3, the server aggregates the updated local model parameters of each edge device in the received training device set according to the following formula: in, These are the parameters of the global model. For training equipment set, This refers to the serial number of the edge device in the training equipment cluster. Indicates the set of training devices in the current global training round. The sum of the number of samples in the local dataset of each edge device. Indicates the first The number of samples in the local dataset of each edge device. After local training in this round of global training, the first Parameters of the local model for each edge device.

Images