A method for accelerating convergence of a global federated learning model and a federated learning system

Abstract

The application relates to a method for accelerating convergence of a global federated learning model and a federated learning system, and belongs to the technical field of industrial Internet of Things.The method comprises the following steps: S1, modeling and analyzing the time delay of the federated learning system; S2, modeling and analyzing the energy consumption of the federated learning system; S3, determining an optimization target; S4, constructing an image classification network and a reinforcement learning intelligent agent network; S5, designing a loss function of the image classification network; S6, converting a node selection problem into a Markov decision process; S7, training the reinforcement learning intelligent agent network; and S8, using a Q network to guide device selection of the federated learning.The federated learning system comprises a cloud server, multiple edge devices and a wireless network.Through the reinforcement learning intelligent agent, the application assists the federated learning system in selecting appropriate devices to participate in training, accelerates the convergence speed of the global federated learning model, reduces the required communication rounds of the federated training, and reduces the communication and energy consumption costs.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial Internet of Things, and particularly relates to a method for accelerating convergence of a global federated learning model and a federated learning system. BACKGROUND

[0002] Industrial Internet of Things introduces a large number of artificial intelligence technologies, aiming to realize data-driven machine learning solutions in heterogeneous and large-scale networks. However, due to the limitations of privacy protection, laws and regulations, etc., a "data island" is formed between devices and institutions. In order to break the data island and fully utilize the data scattered on the devices, federated learning technology is proposed and used in the field of data privacy protection. Federated learning is a distributed machine learning framework that protects user privacy and completes joint modeling without sharing data. The main idea is to train a local model on the edge device, perform federated averaging algorithm on the center cloud server to complete model aggregation, and distribute the aggregated model to all participating federated training edge devices for the next round of local training. The edge device and the cloud server repeat the above process until the global model reaches the target accuracy. However, traditional federated learning faces the following challenges:

[0003] (1) Device heterogeneity: Different edge devices have different computing capabilities, storage space, battery capacity, etc.

[0004] (2) Data heterogeneity: The data collected on mobile devices is usually non-identically distributed, which violates the basic assumption of distributed optimization.

[0005] (3) Unstable network state: Edge devices have limited communication resources, and the data transmission rate is affected by the environment and fluctuates greatly, which may increase the communication cost and transmission power consumption of the federated training process.

[0006] The computing and communication resources of industrial Internet of Things (IIoT) devices are limited, and many IIoT devices are powered by batteries, so the communication and energy consumption costs of the devices need to be considered. In addition, due to the unstable communication quality of IIoT devices, the devices may need to spend a lot of time on local model transmission, increasing the communication cost. Moreover, the Non-IID distribution of data will slow down the convergence speed of the global model, reduce the model accuracy, and cause more communication rounds to achieve model convergence, so accelerating the convergence of the global federated model is of great significance to reduce the communication and energy consumption costs of training.

[0007] Li et al. proposed the FedProx algorithm in their published paper "Federated Optimization in Heterogeneous Networks", which balances the optimization difference between global and local objectives by using a regularization term to reduce the impact of Non-IID data distribution, but does not consider the impact on system communication cost and energy cost in dynamic network scenarios.

[0008] WEN et al. proposed in their published paper "Communication-Efficient Federated Data Augmentation on Non-IID Data" (Conference on Computer Vision and Pattern Recognition Workshops, 2022) to use an autoencoder to generate missing samples for devices, reducing the degree of Non-IID for device datasets, but this method introduces additional computational burden for IIoT devices.

[0009] YANG et al. proposed in their published paper "Improving Accuracy and Convergence in Group-based Federated Learning on Non-IID Data" (IEEE Transactions on Network Science and Engineering, 2022) to group and cluster local models of edge devices, and randomly select devices from each group to participate in federated training during training, but this method cannot determine the optimal number of clusters, which may affect the convergence of the global model. SUMMARY

[0010] To solve the above problems existing in the prior art, the present application provides a method for accelerating the convergence of a global federated learning model and a federated learning system, which improves the convergence speed of the global federated model and reduces the communication and energy cost of federated learning.

[0011] The object of the present application can be achieved by the following technical solutions:

[0012] The present application provides a method for accelerating the convergence of a global federated learning model, comprising the following steps:

[0013] S1, modeling and analyzing the time delay of the federated learning system;

[0014] S2, modeling and analyzing the energy consumption of the federated learning system;

[0015] S3, determining the optimization target;

[0016] S4, constructing an image classification network and a reinforcement learning agent network;

[0017] S5, designing a loss function of the image classification network;

[0018] S6, converting the node selection problem into a Markov decision process;

[0019] S7, training the reinforcement learning agent network;

[0020] S8, using the Q network to guide the device selection of federated learning;

[0021] The reinforcement learning agent network adopts a DDQN reinforcement learning algorithm; the reinforcement learning agent network includes the Q network and a Target Q network, and the Q network and the Target Q network adopt the same network structure.

[0022] Further, the time delay of the federated learning system is modeled and analyzed, and specifically includes:

[0023] The time consumed by the computing device i for performing model training in the kth communication round

[0024]

[0025] Wherein, d i is the number of samples of device i, c i is the number of cpu cycles required for training one sample, f i is the working frequency, and τ local iteration is the round of local iteration.

[0026] The transmission time of the model

[0027]

[0028] Wherein, represents the absolute value of the local model network parameter, and the unit is Byte, represents the data transmission rate of device i in the kth communication round.

[0029] The local training time delay of device i in the kth communication round

[0030]

[0031] The training time delay T of the kth communication process is calculated k :

[0032]

[0033] Further, the energy consumption of the federated learning system is modeled and analyzed, specifically including:

[0034] The energy consumption of the local training performed by the computing device i in the kth communication round;

[0035] The communication power consumption of the model transmission performed by the computing device i;

[0036] The idle energy consumption of the device i is obtained by combining the energy consumption and the communication power consumption;

[0037] The total energy consumed by the computing device is calculated according to the idle energy consumption.

[0038] Further, the optimization objective is described as:

[0039] min (Loss (x; θ)) (1)

[0040] f min ≤f i ≤f max (2)

[0041]

[0042]

[0043] T k <T max (5)

[0044] In the above formula, represents a decision variable, when represents that the device i is selected to participate in federated training in the kth communication round, when represents that the device i is not selected, f min is the minimum value of the device operating frequency, f max is the maximum value of the device operating frequency, B is the server bandwidth, and N is the total number of devices;

[0045] Wherein, formula (1) represents the optimization objective, which represents minimizing the loss function of the global model on the test set, x represents the test set sample, and θ represents the global model parameter; formula (2) represents the constraint on the device operating frequency; formula (3) represents that the total bandwidth of the selected devices is not greater than the server bandwidth B; formula (4) represents that at least one device is selected to participate in federated training in one communication round, and the maximum number does not exceed the total number of devices N; and formula (5) represents that the training delay T k of the kth communication round cannot exceed the specified maximum delay T max .

[0046] Further, the image classification network is a two-layer MLP network; specifically including an image input layer, a first linear network, a first activation function layer, a second linear network, a second activation function layer and a fully connected network.

[0047] Further, the loss function of the designed image classification network specifically includes:

[0048] The loss function of the image classification network includes a cross-entropy loss function (l ce ) and a maximum mean difference loss (l MMD );

[0049] Wherein, l ce is used for image classification tasks; l MMD is used to measure the output difference of the global model and the local model under the input of the local data sample;

[0050] l ce +l MMD as the loss function of the local network model, and gradient descent is performed to update the local model parameters.

[0051] Further, the Markov decision process includes system state, action space, policy, reward function and adjacent state;

[0052] Wherein, the policy represents the mapping from the state space to the action space;

[0053] The reward function is set to minimize the weighted sum of delay and energy consumption.

[0054] Further, the training of the reinforcement learning agent network specifically includes:

[0055] Calculate the time series difference target;

[0056] Define a loss function according to the time series difference target;

[0057] Minimize the loss function by gradient descent method to update the network parameters of the Q network.

[0058] Further, the device selection using the Q network to guide the federated learning specifically includes:

[0059] The cloud server sends the global model parameters to all edge devices participating in the federated learning and collects the state information of the federated learning system;

[0060] Input the state information into the Q network, and the Q network outputs the Q value, which represents the value of each action, and the top-k Q values correspond to the best device subset in the current state;

[0061] The best device subset uses local data to train an image classification network to update a local model, and then uploads the local model to a cloud server;

[0062] The cloud server executes a model aggregation algorithm to update a global model;

[0063] The above process is continuously performed until the global model reaches a target accuracy.

[0064] The present application also provides a federated learning system for accelerating the convergence of a global federated learning model, comprising a cloud server, a plurality of edge devices and a wireless network;

[0065] The cloud server is used to store and update global model parameters, receive and send messages, and run a reinforcement learning agent network and execute a model aggregation algorithm;

[0066] The edge device is used to store and process local data, calculate node scores, perform local training, receive and send messages;

[0067] The wireless network is used to connect the cloud server and the edge device.

[0068] The present application has the following advantages:

[0069] (1) The reinforcement learning agent assists the federated learning system in selecting appropriate devices for training, with the minimum global model loss function on the test data set as the optimization target, to accelerate the convergence speed of the global model and reduce the communication and energy consumption cost of the federated training process.

[0070] (2) The reinforcement learning agent continuously improves the node selection strategy through a large amount of interaction data, making the method have high accuracy and high robustness.

[0071] (3) The reinforcement learning agent is run on the cloud server, which does not introduce additional computational burden to the edge device. BRIEF DESCRIPTION OF DRAWINGS

[0072] In order to facilitate the understanding of those skilled in the art, the present application will be further described below with reference to the accompanying drawings.

[0073] Figure 1 The method flowchart of the present application is shown;

[0074] Figure 2 The structure diagram of the image classifier in the present application is shown;

[0075] Figure 3 The structure diagram of the Q network in the present application is shown;

[0076] Figure 4 The loss function composition diagram of the image classification network in the present application is shown;

[0077] Figure 5 A reward curve chart for a DDQN training phase of an embodiment of the present application;

[0078] Figure 6 A precision comparison curve chart of a simulation experiment of an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the specific embodiments, structures, features and effects thereof according to the present application are described in detail below in combination with the drawings and preferred embodiments.

[0080] A method for accelerating the convergence of a global federated learning model, as shown in Figure 1 , specifically comprising the following steps:

[0081] S1, model and analyze the time delay of the federated learning system.

[0082] In one communication round, the training time delay of the device is composed of the calculation time delay of the local model training and the communication time delay of the model transmission. Assuming that device i has d i samples, the number of cpu cycles required for training one sample is c i , the working frequency is f i , and τ rounds of local iterations are performed, then the time consumed by device i in the kth communication round for model training is:

[0083]

[0084] The local model network parameters trained by device i using local data are uploaded to the parameter server Es, and the transmission time of the model is calculated by the formula:

[0085]

[0086] Wherein, represents the size of the local model network parameters, in Byte, represents the data transmission rate of device i in the kth communication round, The calculation of is calculated by the following Shannon formula:

[0087]

[0088] From the above formula, it can be seen that the data transmission rate of device i is related to the bandwidth B i,k allocated to device i, the transmission power p i,k of device i, and the channel gain g i,kand the noise power N0, the change of the environment state will affect the data transmission rate and exacerbate the uncertainty of the communication delay.

[0089] In combination with the foregoing, the local training delay of the device i in the kth communication round is:

[0090]

[0091] In the federated synchronous learning algorithm, the local training time of each communication round is determined by the slowest device, so the training delay of the kth communication process is:

[0092]

[0093] S2, model the energy consumption of the federated learning system.

[0094] The energy consumption of the device i in the kth communication round for performing local training is:

[0095]

[0096] In the above formula, σ represents the effective capacitance coefficient, which is related to the properties of the chip itself.

[0097] When transmitting the model, the device i uses a power value of p i,k , and the communication power consumption of the device i is:

[0098]

[0099] It should be noted that the device that first completes the local model training and model transmission needs to wait for other devices that have not completed, and the device that executes faster has an idle waiting time. The energy consumed in the idle waiting time is referred to as idle energy consumption. Therefore, the idle energy consumption of the device i is equal to the idle waiting time multiplied by the unit energy consumption in the idle state, and the calculation formula is as follows:

[0100]

[0101] wherein, represents the energy consumed by the device i per unit time in the idle waiting state.

[0102] As can be seen from the above, the energy consumed by the device i in the kth communication round is:

[0103]

[0104] Therefore, in the kth communication round, the total energy consumed by all devices is:

[0105]

[0106] S3, determine the optimization objective.

[0107] The objective is to minimize the loss function of the global model on the test set under dynamic network scenarios, considering the heterogeneity of Industrial Internet of Things (IIoT) devices, and is described using the following formula:

[0108] min (Loss(x; 0)) (1)

[0109] f min ≤f i ≤f max (2)

[0110]

[0111]

[0112] T k <T max (5)

[0113] In the above formula, is a decision variable, when is selected to participate in federated training in the kth communication round, when is not selected, f min is the minimum value of the device working frequency, f max is the maximum value of the device working frequency, B is the server bandwidth, and N is the total number of devices.

[0114] Where formula (1) is the optimization objective, which represents the accuracy of the global model on the test set, formula (2) represents the constraint on the device working frequency, formula (3) represents that the total bandwidth of the selected devices is not greater than the server bandwidth B, formula (4) represents that at least one device is selected to participate in federated training in one communication round, and the maximum number does not exceed the total number of devices N, and formula (5) represents that the training delay T k of the kth communication round cannot exceed the specified maximum delay T max .

[0115] S4, construct an image classification network and a reinforcement learning agent network.

[0116] As Figure 2As shown, by constructing a two-layer MLP (multi-layer perceptron) network as the local image classification network of the IIoT device, an image input layer, a first linear network, a first activation function layer, a second linear network, a second activation function layer and a fully connected network are included; the input dimension of the first linear network is set to 784, the output layer dimension is 200, the input dimension of the second linear network is 200, the output dimension is 200, the input dimension of the fully connected layer is 200, and the output dimension is 10; the first and second activation function layers both adopt ReLU function to realize.

[0117] The reinforcement learning agent network adopts a DDQN (Double Deep Q Network) reinforcement learning algorithm, including a Q network and a Target Q network, and the Q network and the Target Q network adopt the same network structure.

[0118] Further, the network structure of the Q network and the Target Q network is composed of two linear networks, as shown in Figure 3 The input dimension of the first linear network is the state dimension of the federated learning system, and the output dimension is set to 128, and the output of the first linear network is connected to the ReLU activation function.

[0119] In the experiment, 20 IIoT devices are used, and the data transmission rate, working frequency, signal transmission power and sample number of each device at the current time are used as state information, so the state dimension of the federated learning system is 80.

[0120] S5, design the loss function of the image classification network.

[0121] As shown in Figure 4 The loss function of the image classification network is composed of two parts: one part is the cross-entropy loss function (l ce ) for the image classification task; the other part is the maximum mean difference loss (l MMD ) for measuring the output difference of the global model and the local model under the input of the local data sample. ce + l MMD as the loss function of the local network model, and gradient descent is performed to update the local model parameters.

[0122] S6, convert the node selection problem into a Markov decision process.

[0123] The method using reinforcement learning to solve the node selection problem of federated learning first needs to abstract this problem as a Markov decision process. A Markov decision process includes system state S(t), action space A(t), policy π, reward function r, and adjacent state S(t+1), which are as follows:

[0124] The system state S(t) is composed of the data transmission rate β(t) between the device and the server, the working frequency ζ(t) of the device, the signal transmission power T p (t) of the device, and the number of samples ψ(t) owned by the device. Therefore, the system state at time slot t can be defined as:

[0125] S(t) = {β(t), ζ(t), T p (t), ψ(t)}

[0126] The action space A(t) is a vector composed of discrete variables 0-1, which represents the selected state of device i at time t. It represents that the device is selected to participate in the current round of federated training at time t; otherwise, it represents that device i is in an unselected state, and device i can perform other tasks at this time.

[0127] The policy π represents the mapping from the state space S(t) to the action space A(t), i.e., A(t) = π(S(t)). The goal of DRL (deep reinforcement learning) is to learn an optimal policy π, so that the action made by the agent according to the current state can obtain the maximum expected reward.

[0128] The reward function is set in accordance with the optimization goal, i.e., to minimize the weighted sum of the delay and energy consumption, and therefore, the reward function r is represented as:

[0129] r = -Loss(x; θ)

[0130] The adjacent state S(t+1) is determined by the current state S(t) and the policy π, and the specific expression is as follows:

[0131] S(t+1) = S(t) + π(S(t)).

[0132] S7, training a reinforcement learning agent network.

[0133] The DDQN reinforcement learning algorithm updates the Q network parameters during training, and keeps the Target Q network parameters unchanged. After a certain number of iterations, the parameters of the Q network are copied to the Target Q network, thereby avoiding estimation errors and overestimation problems.

[0134] Suppose the network parameters of the Q network are denoted as θ, and the network parameters of the target network are denoted as θ -The calculation of the time-difference target is as follows:

[0135]

[0136] In the above formula, r represents the instant reward returned by the system, s' represents the system state at the next moment, a' represents the action taken, A represents the action space, represents the action taken to obtain the maximum Q value using the Q network for the next state, represents the value estimated using the Target Q network according to the state at the next moment and the action estimated by the Q network.

[0137] The loss function l(θ) is defined as (Y target -Q(s,a;θ)) 2 is updated by backpropagation of the loss function, where s and a represent the current moment and the action taken at the current moment, respectively. The loss function l(θ) is minimized by the gradient descent method, and the update process of θ is as follows:

[0138]

[0139] In the above formula, θ t+1 represents the updated Q network parameter, θ t represents the parameter before updating, represents the gradient of the parameter θ t , s t and a t represent the state at time t and the action taken, respectively. α represents the step size of the update.

[0140] In this embodiment, α is set to 0.001, the update frequency of the Target Q network is 20 times of the Q network, and the Target Q network updates the parameter once using the Q network.

[0141] The DDQN reinforcement learning algorithm updates the parameters of the Q network according to the loss function constructed by the interactive data of the federated learning system and the Target Q network, so that the expected reward obtained reaches convergence.

[0142] S8, using the Q network to guide the device selection of federated learning.

[0143] The cloud server sends global model parameters to all edge devices participating in federated learning and collects state information of the federated learning system; the state information is input into the trained Q network, and the Q network outputs the value of each action, that is, the Q value. The devices corresponding to the top k Q values are used as the best device subset in the current state; the best device subset uses local data to update the received global model parameters locally, and then uploads the updated model parameters to the server, and the server executes a model aggregation algorithm to update the global model; the above process is continuously executed until the global model reaches the target accuracy.

[0144] The application also provides a federated learning system for accelerating convergence of a global federated learning model, comprising a cloud server, multiple edge devices and a wireless network environment. The cloud server is used to store and update global model parameters, receive and send messages, run a reinforcement learning agent network and execute a model aggregation algorithm. The edge devices are used to store and process local data, calculate node scores, perform local training, receive and send messages. The wireless network environment is used to connect the cloud server and the edge devices.

[0145] The effect of the application will be further described below in combination with a simulation experiment:

[0146] (1) Simulation experiment conditions:

[0147] The hardware platform of the simulation experiment is: the processor is Intel(R) Core i7-12700H CPU, the memory is 16GB, and the graphics card is NVIDIA GeForce RTX 3060.

[0148] The software platform of the simulation experiment is: win11 operating system, python 3.9.12, PyTorch1.12.1.

[0149] (2) Simulation content and simulation result analysis:

[0150] The MNIST dataset is used in the simulation experiment of the application, and the IID dataset and the Non-IID dataset are constructed based on the MNIST dataset. The IID dataset is randomly sampled 50 times for each class of samples, consisting of a total of 500 samples. The Non-IID dataset is composed of primary class samples and secondary class samples, wherein the primary class samples account for 70% of the total number of samples, and the remaining 30% of the secondary class samples are uniformly sampled. In the experiment, 70% of the devices are allocated the Non-IID dataset, and the remaining 30% of the devices are allocated the IID dataset. Figure 5 The reward obtained by the reinforcement learning agent in the training stage is shown in the change curve with the number of iterations, and with the increase of the number of iterations, the reward obtained by the DDQN agent in the training process gradually increases with the number of iterations, and tends to converge after 60 iterations.

[0151] The node selection strategy proposed in the present application is to guide the node selection of federated learning through the Q network. In the following description of the simulation experiment, the node selection strategy is referred to as the LCNSFL-2 algorithm, and the two algorithms used for comparison are the Random Selection algorithm and the FedProx algorithm. The number of communication rounds required to reach the target accuracy, the communication cost, the energy cost, and the weighted cost of communication and energy consumption are compared, as shown in Table 1. The target accuracy is set to 90%, and the experimental results show that the LCNSFL-2 algorithm requires the least number of communication rounds, and its communication, energy consumption, and weighted cost are much lower than those of the other two algorithms.

[0152] Table 1 Performance comparison of three algorithms

[0153] Algorithm name Dataset Communication rounds Communication cost Energy cost Weighted cost LCNSFL-2 MNIST 18 3023.7 16175.0 9599.3 Random Selection MNIST 26 4116.3 25221.7 14669.0 FedProx MNIST 22 3549.5 19381.5 11465.5

[0154] From the above and Figure 6 It can be concluded that the LCNSFL-2 algorithm can reach the target accuracy faster, reduce the number of communication rounds with the server, and thus reduce the communication and energy consumption costs of the system.

[0155] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Although the present application has been disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content to obtain equivalent embodiments with equivalent changes, without departing from the scope of the technical solution of the present application. Any modification, equivalent change, and modification of the above embodiments, which does not depart from the technical solution of the present application, is still within the scope of the present application.

Claims

1. A method for accelerating convergence of a global federated learning model, characterized in that: The method comprises the following steps: S1, modeling and analyzing the time delay of the federated learning system; S2, modeling and analyzing the energy consumption of the federated learning system; S3, determining an optimization target; S4, constructing an image classification network and a reinforcement learning agent network; S5, designing a loss function of the image classification network; S6, converting the node selection problem into a Markov decision process; S7, training the reinforcement learning agent network; S8, using the Q network to guide the device selection of the federated learning; The reinforcement learning agent network adopts a DDQN reinforcement learning algorithm; the reinforcement learning agent network comprises the Q network and a Target Q network, and the Q network and the Target Q network adopt the same network structure; The modeling and analyzing of the time delay of the federated learning system specifically comprises: Computing device Time consumed to perform model training in the kth communication round : wherein, the number of samples for the device the number of samples, the number of cpu cycles needed to train one sample, the working frequency, the number of rounds of local iterations; Transmission time of a computational model : wherein, represents an absolute value of the local model network parameter, in Byte, represents the device a data transmission rate in the kth communication round. Computing device Local training latency in the kth communication round : Computing a training latency for the kth communication procedure : ； The optimization target is described as: （1） （2） （3） （4） （5） In the above formula, denotes the decision variable, when denotes that in the kth communication round, the device i is selected to participate in federated training, when denotes that the device i , is the minimum value of the working frequency of the device, is the maximum value of the working frequency of the device, is the server bandwidth, is the total amount of devices; wherein formula (1) represents an optimization objective, which represents minimizing the loss function of the global model on the test set, x represents a test set sample, represents a global model parameter, formula (2) represents a constraint on the working frequency of the device, formula (3) represents that the total bandwidth of the selected device is not greater than the server bandwidth , formula (4) represents that at least one device is selected to participate in federated training in one communication round, and the maximum number does not exceed the total number of devices , formula (5) represents that the training delay Tk in the kth communication round cannot exceed the specified maximum delay Tmax. The using of the Q network to guide the device selection of the federated learning specifically comprises: The cloud server sends global model parameters to all edge devices participating in the federated learning and collects state information of the federated learning system; The state information is input into the Q network, and the Q network outputs Q values, wherein the Q values represent the values of each action, and the devices corresponding to the top-k Q values are used as the best device subset in the current state; The best device subset trains the image classification network using local data to update the local model, and then uploads the local model to the cloud server; The cloud server executes a model aggregation algorithm to update the global model; The above process is continuously executed until the global model reaches the target accuracy.

2. The method of claim 1, wherein the method further comprises: The modeling and analyzing of the energy consumption of the federated learning system specifically comprises: Computing device Energy consumption for performing local training at the kth communication round; Computing device Communication power consumption for model transfer; combining the energy consumption with the communication power consumption to obtain an idle energy consumption of the device ; The total energy consumed by the device is calculated according to the idle energy consumption.

3. The method of claim 1, wherein: The image classification network is a two-layer MLP network, which specifically comprises an image input layer, a first linear network, a first activation function layer, a second linear network, a second activation function layer, and a fully connected network.

4. The method of claim 1, wherein: The designing of the loss function of the image classification network specifically comprises: The loss function of the image classification network comprises a cross-entropy loss function and a maximum mean discrepancy loss ; wherein, for an image classification task; for measuring the difference between the outputs of the global model and the local model under the input of the local data samples; will be described below. As the loss function of the local network model, and gradient descent is performed thereon to update the local model parameters.

5. The method of claim 1, wherein: The Markov decision process comprises a system state, an action space, a policy, a reward function, and adjacent states; The policy represents a mapping from the state space to the action space; The reward function is set as a weighted sum of the minimized time delay and energy consumption.

6. The method of claim 1, wherein: The training of the reinforcement learning agent network specifically comprises: Calculating a time difference target; Defining a loss function according to the time difference target; Minimizing the loss function by the gradient descent method to update the network parameters of the Q network.

7. The system for accelerating convergence of a global federated learning model according to any one of claims 1-6, wherein: The system comprises a cloud server, a plurality of edge devices, and a wireless network; The cloud server is configured to store and update global model parameters, receive and send messages, and run a reinforcement learning agent network and execute a model aggregation algorithm; The edge devices are configured to store and process local data, calculate node scores, perform local training, and receive and send messages; The wireless network is configured to connect the cloud server and the edge devices.

Images