Federal learning-based time delay and energy consumption optimization method

Abstract

The invention relates to the field of artificial intelligence, and particularly discloses a time delay and energy consumption optimization method based on federated learning, and the method comprises the steps: enabling a terminal layer, an edge layer and a cloud layer to work cooperatively: the terminal layer is responsible for collecting data and extracting features through a convolutional neural network; the edge layer receives the features and trains an edge model, and then uploads parameters to the cloud layer; and the cloud layer aggregates the parameters to form a global model and sends the global model back to the edge layer for the next round of training. In addition, a global time delay and energy consumption model is also constructed, and optimization is carried out through a particle swarm optimization algorithm, so that the time delay and energy consumption in the training process are further reduced, and the efficiency and performance of model training are improved. According to the method, resource utilization is optimized, and a more efficient model training process is realized by optimizing time delay and energy consumption of the terminal layer, the edge layer and the cloud layer.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for optimizing delay and energy consumption based on federated learning. Background Art

[0002] As the main source of network data in the future, IoT devices equipped with advanced sensors are ubiquitous. The collection of massive data samples naturally migrates from traditional cloud servers to edge IoT devices. These data samples are the basis for data-driven artificial intelligence model training. However, IoT devices with limited computing resources face model learning performance bottlenecks and energy consumption costs, and are unable to independently complete local model training tasks. Therefore, hierarchical federated learning algorithms came into being.

[0003] Federated Learning (FL) has become a distributed collaborative artificial intelligence method that allows distributed IoT devices to collaborate on training machine learning models without disclosing private data to other devices or servers, thereby realizing intelligent IoT applications without data sharing. Although federated learning solves the problem of building training models for IoT devices, the existing federated learning framework still has the problem of excessive communication overhead. This is because the model parameters need to be frequently trained during the interaction and aggregation process, and data communication poses a great challenge to communication bandwidth. Therefore, many scholars have proposed a federated learning architecture based on model compression, a model aggregation method based on a segmentation learning mechanism, and a federated learning method based on gradient aggregation for resource-constrained IoT devices during the training process. It can be seen that federated learning solves the problems of uneven data distribution and secure data transmission through a model parameter sharing algorithm. However, for IoT devices, the existing local model parameter aggregation method poses a great challenge to the communication of IoT devices. Summary of the invention

[0004] The embodiment of the present invention provides a method for optimizing latency and energy consumption based on federated learning. The method is composed of three layers: the terminal layer, the edge layer, and the cloud layer working together: the terminal layer is responsible for collecting data and extracting features through a convolutional neural network; the edge layer receives these features and trains the edge model, and then uploads the parameters to the cloud layer; the cloud layer aggregates these parameters to form a global model and sends it back to the edge layer for the next round of training. In addition, the present invention also constructs a global latency and energy consumption model, and optimizes it through a particle swarm algorithm to further reduce latency and energy consumption during training and improve the efficiency and performance of model training.

[0005] The present invention provides a method for optimizing delay and energy consumption based on federated learning, comprising:

[0006] The terminal layer collects data, extracts feature data using a convolutional neural network, and then uploads the feature data to the edge layer, while determining the latency and energy consumption required for extracting and uploading the feature data;

[0007] After receiving the feature data, the edge layer performs convolutional neural network training to obtain the edge model, and uploads the parameters of the edge model to the cloud layer, while determining the latency and energy consumption required for convolutional neural network training;

[0008] After receiving the edge model parameters, the cloud layer aggregates the edge model parameters and determines the latency and energy consumption required for parameter aggregation;

[0009] The cloud layer sends the aggregated edge model parameters to the edge layer. After the edge layer obtains the aggregated edge model parameters, it updates the edge model and prepares for the next round of training.

[0010] According to the latency and energy consumption required for extracting and uploading feature data at the terminal layer, training convolutional neural networks at the edge layer, and aggregating parameters at the cloud layer, a global latency and energy consumption model is constructed. The particle swarm algorithm is used to optimize the global latency and energy consumption model to reduce training latency and energy consumption.

[0011] Furthermore, the extraction of the feature data includes:

[0012] After extracting feature data using convolutional neural network, the hidden layer features obtained are expressed as:

[0013] a t =tanh(w ax x t +w aa a t-1 +b a )

[0014] Among them, a t represents the hidden layer features, x t represents the information data of the terminal layer at time t, w ax Represents the link weight from the information data input layer to the hidden layer, w aa Represents the previous layer of implicit features a t-1 The link weight from the input layer to the hidden layer, b a Indicates bias.

[0015] Furthermore, the convolutional neural network training includes:

[0016] Use the received feature data for training;

[0017] Among them, the i+1 round model training process of the kth base station server is expressed as:

[0018]

[0019] In the above formula, η represents the learning rate of the edge model, represents the training model of the base station server in the i-th round, represents the training model of the base station server in the i+1th round, Represents the gradient of the loss function for model training.

[0020] Furthermore, the parameter aggregation includes:

[0021] receiving edge model parameters from edge layer training;

[0022] According to the rules, the number of aggregations of the corresponding edge model is selected to perform parameter aggregation;

[0023] The edge model parameter aggregation process of N base stations is expressed as:

[0024]

[0025] In the above formula, Δw represents the global parameter of the cloud model, and |D| represents the size of all data sets of the edge models of N base stations.

[0026] Furthermore, the delay required for extracting and uploading the feature data specifically includes the feature data extraction delay and the feature data upload delay. The delay required for extracting and uploading the feature data is The extraction delay of feature data is The characteristic data upload delay is The corresponding minimum delay objective function is expressed as:

[0027]

[0028] st

[0029]

[0030] In the above formula, in the IoT device set S (S = {1, 2, ..., N}), X represents the channel allocation strategy between the base station and the IoT device, and the element x in X is n,c,t Indicates the channel allocation result. Indicates that at time t, channel c is allocated to IoT device S n ;

[0031] Among them, the first constraint means that at any time t, all IoT devices are assigned to at most C channels;

[0032] The second constraint states that at any time t, any IoT device can be assigned at most one channel;

[0033] The third constraint states that any IoT device Sn During channel availability time Internal feature information The transmission is completed;

[0034] The fourth constraint states that any IoT device S n The latency required for data processing is equal to the latency of feature extraction by IoT devices. Delay in uploading features to IoT devices sum;

[0035] The fifth constraint states that any IoT device S n After the feature data of the collected data is extracted locally, upload it;

[0036] The sixth constraint states that the transmission rate of IoT devices is limited by the bandwidth W of channel c. c 、IoT Devices n Its own transmission power and Gaussian noise N s The impact of

[0037] The delay required for the convolutional neural network training is the training delay of the base station server set, which is expressed as:

[0038]

[0039] In the above formula, It is represented as the training delay of the base station server set BS (BS = 1, 2, ..., M), m1 represents the number of iterations of the base station server training edge model, and f0 represents the number of CPU cycles required for the base station server to train 1 bit of data. Indicates the available computing power of the kth base station server when training the edge model, represents all data sets of the kth base station server;

[0040] The delay required for parameter aggregation is the delay required for the cloud layer to aggregate the parameters of the edge model, which is expressed as:

[0041]

[0042] Among them, t Cloud is the time delay required for the cloud layer to aggregate the parameters of the edge model, q represents the number of quantization bits of each edge model parameter, and V represents the number of all edge model parameters. It represents the compression rate when the edge model is selected in the cloud layer aggregation process, and R represents the uplink data transmission rate of each base station.

[0043] Furthermore, the total delay to complete a model aggregation is expressed as:

[0044]

[0045] In the above formula, T is the total delay to complete a model aggregation. The delay required for extracting and uploading feature data, Denotes the time delay required for convolutional neural network training, t Cloud is the delay required for parameter aggregation.

[0046] Furthermore, the energy consumption required for extracting and uploading the characteristic data includes the energy consumption for extracting the characteristic data and the energy consumption for uploading the characteristic data, wherein the energy consumption for extracting the characteristic data is expressed as:

[0047]

[0048] In the above formula, Represents any IoT device S within the coverage of the kth base station server n Energy consumption of extracting features, Represents IoT device S n The size of all collected data sets, f0' represents the number of CPU cycles required for IoT devices to extract 1-bit data features, Represents IoT device S n The available computing power when extracting features, ξ' represents the effective conversion capacity related to the chip structure of the IoT device;

[0049] The energy consumption of uploading the characteristic data is expressed as:

[0050]

[0051] In the above formula, Represents any IoT device S within the coverage of the kth base station server n Energy consumption of uploaded data characteristics, The number of quantized bits representing each feature data, Indicates the number of feature data. Indicates the transmit power of IoT devices, Indicates the characteristic data uplink rate;

[0052] The energy consumption required for the convolutional neural network training is the training energy consumption of the base station server set, which is expressed as:

[0053]

[0054] In the above formula, represents the kth base station server BS kEnergy consumption of training edge model, m1 represents the number of iterations of edge model training on base station server, f0 represents the number of CPU cycles required for base station server to train 1 bit of data, Indicates the available computing power of the kth base station server when training the edge model, represents all data sets of the kth base station server, ξ represents the effective conversion capacity related to the chip structure of the base station server;

[0055] The energy consumption required for parameter aggregation is the energy consumption required by the cloud layer to aggregate the parameters of the edge model, which is expressed as:

[0056]

[0057] In the above formula, represents the energy consumption of the cloud model aggregating the edge model parameters after the kth base station server sends the edge model parameters to the cloud via wired mode, Indicates the number of quantization bits for each edge model parameter, represents the number of edge model parameters, represents the transmission power of the base station, Indicates the uplink rate of edge model parameters, Indicates the compression ratio when the edge model is selected during the cloud-layer aggregation process.

[0058] Furthermore, the total energy consumption to complete a model aggregation is expressed as:

[0059]

[0060] In the above formula, E represents the total energy consumption to complete a model aggregation. represents the energy consumption of extracting feature data, Indicates the energy consumption of uploading characteristic data, represents the energy consumption required for convolutional neural network training, Represents the energy consumption required for parameter aggregation.

[0061] Furthermore, the total delay energy consumption to complete a model aggregation is expressed as:

[0062] min P = γ E E+γ T T

[0063] In the above formula, P is the total delay energy consumption to complete a model aggregation, E is the total energy consumption to complete a model aggregation, T is the total delay to complete a model aggregation, γ E and γ T is the normalized parameter of aggregation delay and energy consumption, which is and

[0064] Furthermore, the total delay energy consumption for completing a model aggregation is optimized using a particle swarm algorithm. Each particle in the swarm updates its position and speed to find the optimal solution in the solution space. The particle position update is expressed as:

[0065]

[0066] The particle velocity update is expressed as:

[0067]

[0068] In the above formula, denote the d-th dimension component of the position vector of particle i in the k-th and k+1-th iterations respectively, denote the d-th dimension component of the velocity vector of particle i in the k-th and k+1-th iterations, respectively; c1 and c2 are the individual and group learning factors, respectively; w is the inertia weight, represents the local optimum, represents the global optimal value, r1 and r2 are random numbers.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] The present invention specifically discloses a method for optimizing delay and energy consumption based on federated learning, which is composed of three layers: the terminal layer, the edge layer and the cloud layer working together: the terminal layer is responsible for collecting data and extracting features through a convolutional neural network; the edge layer receives these features and trains the edge model, and then uploads the parameters to the cloud layer; the cloud layer aggregates these parameters to form a global model and sends it back to the edge layer for the next round of training. In addition, the present invention also constructs a global delay and energy consumption model, and optimizes it through a particle swarm algorithm to further reduce the delay and energy consumption during the training process, and improve the efficiency and performance of model training. This method solves the problems of excessive communication overhead and data security of the Internet of Things network by splitting the model into multiple smaller networks and adopting a cross-client relay training method. In addition, with the help of the powerful computing and communication resources of the edge computing network, the performance of machine learning tasks in Internet of Things devices can be greatly improved, ensuring the accuracy and communication efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a flowchart of a federated learning method in an edge computing network provided by an embodiment of the present invention;

[0072] Figure 2 It is a schematic diagram of a three-layer federated learning model of a federated learning method in an edge computing network provided by an embodiment of the present invention;

[0073] Figure 3It is a flow chart of a particle swarm algorithm of a federated learning method in an edge computing network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] like Figure 1 , a method for federated learning in an edge computing network provided by an embodiment of the present invention includes steps S1 to S5, involving a method such as Figure 2 The three-layer federated learning model shown includes the terminal layer, edge layer, and cloud layer, referred to as the terminal-edge-cloud model.

[0076] Figure 1 Steps S1 to S5 in the embodiment are as follows:

[0077] S1. The terminal layer collects data, extracts feature data using a convolutional neural network, and then uploads the feature data to the edge layer. At the same time, it determines the latency and energy consumption required for extracting and uploading feature data.

[0078] In a specific embodiment, the terminal layer uses a convolutional neural network to extract features from the collected data. The terminal layer can effectively remove redundant or noisy information by preprocessing the data, making subsequent model training more efficient. After extracting feature data using a convolutional neural network, the hidden layer features obtained are expressed as:

[0079] a t =tanh(w ax x t +w aa a t-1 +b a )

[0080] Among them, a t represents the hidden layer features, x t represents the information data of the terminal layer at time t, w ax Represents the link weight from the information data input layer to the hidden layer, w aa Represents the previous layer of implicit features a t-1 The link weight from the input layer to the hidden layer, b a Indicates bias.

[0081] When the terminal layer completes feature extraction, the feature data is uploaded to the edge layer.

[0082] S2. After receiving the feature data, the edge layer performs convolutional neural network training to obtain the edge model, and uploads the parameters of the edge model to the cloud layer, while determining the latency and energy consumption required for convolutional neural network training.

[0083] In a specific embodiment, the edge layer obtains features and puts them into the base station server for edge model training. The edge computing network of the base station server will perform neural network model training for each IoT device in turn. After the pooling layer, activation layer, and fully connected layer operations, the stochastic gradient descent (SGD) algorithm is used to quickly converge the model parameters. The i+1 round model training process of the kth base station server can be expressed as:

[0084]

[0085] Among them, η represents the learning rate of the edge model, represents the training model of the base station server in the i-th round, represents the training model of the base station server in the i+1th round, Represents the gradient of the loss function for model training.

[0086] When the convolutional neural network training is completed, the edge model parameters obtained after training are transferred to the cloud-layer federated server that executes the global model aggregation algorithm.

[0087] S3. After receiving the edge model parameters, the cloud layer aggregates the edge model parameters and determines the latency and energy consumption required for parameter aggregation.

[0088] The cloud layer obtains global model parameters through parameter aggregation. Cloud layer parameter aggregation refers to the cloud server receiving edge model parameters from edge layer training, selecting the corresponding edge model aggregation times according to the rules, performing model aggregation, and then sending the aggregated cloud model parameters to the edge server. The aggregation times rule of the edge model can be selected according to the following embodiments: In one embodiment, the rule is the aggregation times pre-set by the technician, and no matter how much the accuracy is achieved, it is sufficient to reach the specified aggregation times; in another embodiment, the accuracy is pre-set by the technician, and aggregation is stopped when the preset accuracy is reached.

[0089] In a specific embodiment, the edge model parameter aggregation process of N base stations can be expressed as:

[0090]

[0091] Among them, |D| represents the size of all data sets of the edge models of N base stations, and Δw represents the global parameters of the cloud model.

[0092] Since most edge models do not play a big role in updating the global model, many scholars have compressed the model parameters to reduce the amount of interactive data of the model. The hierarchical federated learning mechanism based on model parameter compression can accelerate the convergence speed of model training to a certain extent. However, since too many model parameter interactions will inevitably produce huge delays and energy consumption, when aggregating model parameters, it is necessary to refer to the status of the base station server to adopt a suitable model compression rate to balance the mutual influence between the convergence speed and communication cost during model training. For example, by selecting v parameters with the largest absolute value to aggregate model parameters, the model compression rate can be adaptively selected to achieve model parameter integration. The model compression rate can be adaptively selected to achieve model parameter integration by comprehensively considering the convergence speed and communication cost of the model.

[0093] However, the computing power and communication capabilities of different IoT devices are different, and the time for the edge layer base station server to synchronize updates in the same round will vary greatly. Therefore, how to balance the federated learning time and energy consumption is the key issue to be solved in this application. The following step S5 will focus on explaining how the learning time and energy consumption are combined with federated learning.

[0094] S4. The cloud layer sends the aggregated edge model parameters to the edge layer. After the edge layer obtains the aggregated edge model parameters, it updates the edge model and prepares for the next round of training.

[0095] The edge layer obtains the parameters sent by the cloud layer and updates the edge model. This process is repeated over and over again to continuously update the edge model and the cloud model.

[0096] S5. Based on the latency and energy consumption required for extracting and uploading feature data at the terminal layer, training convolutional neural networks at the edge layer, and aggregating parameters at the cloud layer, a global latency and energy consumption model is constructed. The particle swarm algorithm is used to optimize the global latency and energy consumption model to reduce training latency and energy consumption.

[0097] In the three-layer federated learning architecture, the three layers of devices cooperate with each other to complete the model training. In model training, the number of devices, computing power and communication capabilities of different layers are different, so different update strategies are required according to the device conditions of this layer. The update process involves three parts: data processing delay, communication delay and energy consumption. Therefore, this article will solve the time and energy consumption problems of federated learning through communication models and energy consumption models.

[0098] In the three-layer architecture of end-edge-cloud, the processing time mainly includes communication delay and data processing delay. Channels covered by the same base station can be reused. There are multiple IoT devices covered by the same base station, and the performance of each IoT device has certain heterogeneity. Since the transmission rate and delay of different channels in the base station are inconsistent, if the data transmission of the channel is not reasonably allocated, once multiple terminals compete for the same channel, the waiting time and transmission time of the IoT device when transmitting data will be very long, resulting in a slow convergence of the model. The update between the IoT terminal layer and the edge layer adopts an asynchronous update strategy. Only by reasonably allocating channels can it be possible to minimize the time required for federated learning. The edge layer and the cloud layer are connected by optical fiber, and the computing power resources of the server are not much limited compared to IoT devices. Therefore, the edge layer and the cloud layer adopt a synchronous update strategy, and the data processing delay is the focus when considering the delay.

[0099] In a specific embodiment, the latency of the three-layer architecture is mainly composed of terminal layer data processing latency, edge layer training latency, and cloud layer parameter aggregation latency.

[0100] Considering the data processing delay at the terminal layer, in the IoT set S (S = {1, 2, ..., N}), X is used to represent the channel allocation strategy between the base station and the IoT terminal. The element x in X n,c,t Indicates the channel allocation result. Indicates that at time t, channel c is allocated to IoT device S n , and vice versa. Since the asynchronous update strategy is adopted between the base station and the IoT terminal, the channel allocation strategy between the base station and the IoT terminal needs to minimize the time required for feature information processing (the processing process includes feature extraction and feature upload) The corresponding objective function is:

[0101]

[0102] st

[0103]

[0104] Among them, the first constraint condition means that at any time t, all IoT devices can be assigned to at most C channels; the second constraint condition means that at any time t, any IoT device can be assigned to at most 1 channel; the third constraint condition means that any IoT terminal S n During channel availability time Internal feature information The transmission is completed; the fourth constraint condition indicates that any IoT terminal S n The delay required for data processing is equal to the delay of feature extraction by the IoT terminal Delay in uploading features to IoT terminals The fifth constraint condition indicates that any IoT terminal S n The data can only be uploaded after the features of the collected data are extracted locally; the sixth constraint condition indicates that the transmission rate of the IoT terminal is subject to the bandwidth W of the channel c. c 、IoT Terminal S n Its own transmission power and Gaussian noise N s impact.

[0105] Considering the edge layer training delay, after receiving the feature data, the base station server inputs the feature into the neural network of the specified structure, trains the edge model, and uploads the edge model parameters to the cloud for aggregation. Since the base station and the cloud server are connected by optical fiber, the edge layer and the cloud layer adopt a synchronous update strategy. The cloud layer performs weighted averaging on the edge model and then returns the result to the base station server.

[0106] For the base station server set BS (BS = 1, 2, ..., M), the training delay It is expressed as:

[0107]

[0108] Among them, m1 represents the number of iterations of the base station server training edge model, f0 represents the number of CPU cycles required for the base station server to train 1 bit of data, Indicates the available computing power of the kth base station server when training the edge model, Represents the size of all data sets of the kth base station server.

[0109] After training is completed, the model parameters are uploaded to the cloud. Since the base station and the cloud are connected by optical fiber, the delay is negligible compared to the training delay.

[0110] Considering the cloud-layer parameter aggregation delay, the cloud adopts a synchronous model aggregation mechanism, so the edge model parameter aggregation delay can be expressed as:

[0111]

[0112] Where R represents the uplink data transmission rate of each edge layer base station. q represents the number of quantization bits of each model parameter, and V represents the number of all parameters of the edge model. It represents the compression rate when selecting the edge model in the cloud layer aggregation process. The model parameters can be aggregated by selecting v parameters with the largest absolute values, and the model compression rate can be adaptively selected to achieve the integration of model parameters.

[0113] In summary, the latency of completing a model aggregation on the edge-cloud can be expressed as:

[0114]

[0115] In a specific embodiment, the energy consumption of the three-layer architecture is mainly composed of the energy consumption of feature extraction at the terminal layer, the energy consumption of uploading feature data at the terminal layer, the energy consumption of training at the edge layer, and the energy consumption of parameter aggregation at the cloud layer.

[0116] Considering the energy consumption of terminal layer feature extraction, any IoT terminal S under the coverage of the kth base station server n Energy consumption of feature extraction

[0117]

[0118] in, Represents IoT device S n The size of all collected data sets, f0' represents the number of CPU cycles required for IoT devices to extract 1-bit data features, Represents IoT device S n The amount of computing power available when extracting features, ξ' represents the effective conversion capacity related to the chip structure of the IoT device.

[0119] Considering the energy consumption of uploading at the terminal layer, any IoT terminal S under the coverage of the kth base station server n Energy consumption of uploaded data characteristics

[0120]

[0121] in, The number of quantized bits representing each feature data, Indicates the number of feature data. Indicates the transmission power of the IoT terminal. Indicates the characteristic data uplink rate, which is related to the bandwidth, transmission power, channel power gain and noise power allocated to the IoT terminal. The characteristic data uplink rate is related to the bandwidth, transmission power, channel power gain and noise power allocated to the IoT terminal, and can be calculated using Shannon's theorem or Nyquist criterion.

[0122] Considering the edge layer training energy consumption, the kth base station server BS k Energy consumption of training edge models

[0123]

[0124] Among them, m1 represents the number of iterations of the base station server training edge model, f0 represents the number of CPU cycles required for the base station server to train 1 bit of data, represents the available computing power of the kth base station server when training the edge model, represents all data sets of the kth base station server, and ξ represents the effective conversion capacity related to the chip structure of the base station server. The chip structure conversion capacity is related to the chip structure, and the conversion capacity of each architecture has a reference value, such as Intel and ARM, both of which have their own reference values.

[0125] Considering the energy consumption of cloud layer aggregation, after the kth base station server sends the edge model parameters to the cloud via wired mode, the cloud model aggregates the energy consumption of the edge model parameters.

[0126]

[0127] in, Indicates the number of quantization bits for each edge model parameter, represents the number of edge model parameters, represents the transmission power of the base station, Indicates the uplink rate of edge model parameters, Indicates the compression ratio when edge models are selected during the cloud-layer aggregation process.

[0128] Combining the above methods, a device-edge-cloud model aggregation process is implemented. The total energy consumption of completing a model aggregation can be expressed as:

[0129]

[0130] In a specific embodiment, the total latency and energy consumption of completing the aggregation latency and energy consumption of the edge-cloud model is expressed as:

[0131] min P = γ E E+γ T T

[0132] Among them, γ E and γ T is the normalized parameter of aggregation delay and energy consumption, which can be used and express.

[0133] In a specific embodiment, the total delay energy consumption formula for completing a model aggregation is optimized using a particle swarm algorithm, and each particle in the group quickly finds the optimal solution in the solution space by adjusting its position and speed. It is characterized in that the total delay energy consumption for completing a model aggregation is optimized using a particle swarm algorithm, and each particle in the group updates its position and speed to find the optimal solution in the solution space, and the position update of the particle is expressed as:

[0134]

[0135] The particle velocity update is expressed as:

[0136]

[0137] in, They represent the d-th dimension component of the position vector of particle i in the kth and k+1th iterations respectively (the position vector can be understood as state value, d is equal to 4), They represent the d-th dimension of the velocity vector of particle i in the kth and k+1th iterations, respectively. c1 and c2 are the individual and group learning factors (values ​​between 0.6-1.4), respectively, and w is the inertia weight (value between 0.5-0.9). The learning factor adjusts the position of the particle swarm in the search space and determines the optimization result. The inertia weight w has a significant impact on the algorithm optimization. represents the local optimum, represents the global optimal value, r1 represents the random number in the particle swarm algorithm, and its value is between 0 and 1; r2 represents the random number in the particle swarm algorithm, and its value is between 0 and 1

[0138] The particle swarm algorithm process is as follows Figure 3 As shown. First, initialize the particle swarm parameters; then sample the compression rate of the edge model, the available computing power of the IoT device, the transmission power of the IoT terminal, and the rate at which the IoT device transmits data, and calculate the aggregate fitness value corresponding to each particle as the fitness value; then find the local and global optimal fitness values; update the position of the particle swarm according to the formula, and update the local and global optimal fitness values ​​of the population; determine whether the particles are gathered near the global minimum delay and energy consumption, if not, return to the previous step, if so, update the position and optimal fitness value of the particle swarm according to the inertia and learning factor; then determine whether the fitness value reaches the set optimal value, if not, return to the previous step, otherwise maintain the global minimum delay and energy consumption; finally determine whether the delay and energy consumption requirements are met, if not, restart a new round of training, if so, take the global minimum delay and energy consumption, find the location of the particle and end the program.

[0139] The present invention transfers most of the training and aggregation of local model parameters from the IoT device without violating data privacy protection. After the IoT device completes the training of the convolutional layer neural network, the underlying features of the convolutional layer network training are uploaded to the edge computing network. The edge computing network will perform neural network model training for each IoT device in turn. After the pooling layer, activation layer, and fully connected layer operations, the model parameters are quickly converged using the stochastic gradient algorithm, and the parameters obtained after training are transferred to the federal server that executes the global model aggregation algorithm to obtain the global model parameters. This method solves the problems of excessive IoT network communication overhead and data security by splitting the model into multiple smaller networks and adopting a cross-client relay training method. In addition, the powerful computing and communication resources of the edge computing network can greatly improve the performance of machine learning tasks in IoT devices, ensuring the accuracy and communication efficiency of the model.

[0140] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for optimizing latency and energy consumption based on federated learning, characterized in that: The terminal layer collects data, extracts feature data using a convolutional neural network, and then uploads the feature data to the edge layer, while determining the latency and energy consumption required for extracting and uploading the feature data; After receiving the feature data, the edge layer performs convolutional neural network training to obtain the edge model, and uploads the parameters of the edge model to the cloud layer, while determining the latency and energy consumption required for convolutional neural network training; After receiving the edge model parameters, the cloud layer aggregates the edge model parameters and determines the latency and energy consumption required for parameter aggregation; The cloud layer sends the aggregated edge model parameters to the edge layer. After the edge layer obtains the aggregated edge model parameters, it updates the edge model and prepares for the next round of training. According to the latency and energy consumption required for extracting and uploading feature data at the terminal layer, training convolutional neural networks at the edge layer, and aggregating parameters at the cloud layer, a global latency and energy consumption model is constructed. The particle swarm algorithm is used to optimize the global latency and energy consumption model to reduce training latency and energy consumption.

2. The method for optimizing delay and energy consumption based on federated learning according to claim 1, characterized in that: The extraction of the feature data comprises: After extracting feature data using convolutional neural network, the hidden layer features obtained are expressed as: a t =tanh(w ax x t +w aa a t-1 +b a ) Among them, a t represents the hidden layer features, x t represents the information data of the terminal layer at time t, w ax Represents the link weight from the information data input layer to the hidden layer, w aa Represents the previous layer of implicit features a t-1 The link weight from the input layer to the hidden layer, b a Indicates bias.

3. The method for optimizing delay and energy consumption based on federated learning according to claim 1, characterized in that: The convolutional neural network training includes: Use the received feature data for training; Among them, the i+1 round model training process of the kth base station server is expressed as: In the above formula, η represents the learning rate of the edge model, represents the training model of the base station server in the i-th round, represents the training model of the base station server in the i+1th round, Represents the gradient of the loss function for model training.

4. The method for optimizing latency and energy consumption based on federated learning according to claim 1, characterized in that: The parameter aggregation includes: receiving edge model parameters from edge layer training; According to the rules, the number of aggregations of the corresponding edge model is selected to perform parameter aggregation; The edge model parameter aggregation process of N base stations is expressed as: In the above formula, Δw represents the global parameter of the cloud model, and |D| represents the size of all data sets of the edge models of N base stations.

5. A method for optimizing delay and energy consumption based on federated learning as claimed in any one of claims 2 to 4, characterized in that: The time delay required for extracting and uploading the feature data specifically includes the time delay for extracting the feature data and the time delay for uploading the feature data. The time delay required for extracting and uploading the feature data is: The extraction delay of feature data is The characteristic data upload delay is The corresponding minimum delay objective function is expressed as: In the above formula, in the IoT device set S (S = {1, 2, ..., N}), X represents the channel allocation strategy between the base station and the IoT device, and the element x in X is n,c,t Indicates the channel allocation result. Indicates that at time t, channel c is allocated to IoT device S n ; Among them, the first constraint means that at any time t, all IoT devices are assigned to at most C channels; The second constraint states that at any time t, any IoT device can be assigned at most one channel; The third constraint states that any IoT device S n During channel availability time Internal feature information The transmission is completed; The fourth constraint states that any IoT device S n The latency required for data processing is equal to the latency of feature extraction by IoT devices. Delay in uploading features to IoT devices sum; The fifth constraint states that any IoT device S n After the feature data of the collected data is extracted locally, upload it; The sixth constraint states that the transmission rate of IoT devices is limited by the bandwidth W of channel c. c 、IoT devices n Its own transmission power and Gaussian noise N s The impact of The delay required for the convolutional neural network training is the training delay of the base station server set, which is expressed as: In the above formula, It is represented as the training delay of the base station server set BS (BS = 1, 2, ..., M), m1 represents the number of iterations of the base station server training edge model, and f0 represents the number of CPU cycles required for the base station server to train 1 bit of data. Indicates the available computing power of the kth base station server when training the edge model, represents all data sets of the kth base station server; The delay required for parameter aggregation is the delay required for the cloud layer to aggregate the parameters of the edge model, which is expressed as: Among them, t Cloud is the time delay required for the cloud layer to aggregate the parameters of the edge model, q represents the number of quantization bits of each edge model parameter, and V represents the number of all edge model parameters. It represents the compression rate when the edge model is selected in the cloud layer aggregation process, and R represents the uplink data transmission rate of each base station.

6. The method for optimizing delay and energy consumption based on federated learning according to claim 5, characterized in that: The total delay to complete a model aggregation is expressed as: In the above formula, T is the total delay to complete a model aggregation. The delay required for extracting and uploading feature data, Denotes the time delay required for convolutional neural network training, t Cloud is the delay required for parameter aggregation.

7. A method for optimizing delay and energy consumption based on federated learning as claimed in any one of claims 2 to 4, characterized in that: The energy consumption required for extracting and uploading the characteristic data includes the energy consumption for extracting the characteristic data and the energy consumption for uploading the characteristic data, wherein the energy consumption for extracting the characteristic data is expressed as: In the above formula, Represents any IoT device S within the coverage of the kth base station server n Energy consumption of extracting features, Represents IoT device S n The size of all collected data sets, f0' represents the number of CPU cycles required for IoT devices to extract 1-bit data features, Represents IoT device S n The available computing power when extracting features, ξ' represents the effective conversion capacity related to the chip structure of the IoT device; The energy consumption of uploading the characteristic data is expressed as: In the above formula, Represents any IoT device S within the coverage of the kth base station server n Energy consumption of uploaded data characteristics, The number of quantized bits representing each feature data, Indicates the number of feature data, Indicates the transmit power of IoT devices, Indicates the characteristic data uplink rate; The energy consumption required for the convolutional neural network training is the training energy consumption of the base station server set, which is expressed as: In the above formula, represents the kth base station server BS k Energy consumption of training edge model, m1 represents the number of iterations of edge model training on base station server, f0 represents the number of CPU cycles required for base station server to train 1 bit of data, Indicates the available computing power of the kth base station server when training the edge model, represents all data sets of the kth base station server, ξ represents the effective conversion capacity related to the chip structure of the base station server; The energy consumption required for parameter aggregation is the energy consumption required by the cloud layer to aggregate the parameters of the edge model, which is expressed as: In the above formula, represents the energy consumption of the cloud model aggregating the edge model parameters after the kth base station server sends the edge model parameters to the cloud via wired mode, Indicates the number of quantized bits for each edge model parameter, represents the number of edge model parameters, represents the transmission power of the base station, Indicates the uplink rate of edge model parameters, Indicates the compression ratio when the edge model is selected during the cloud-layer aggregation process.

8. The method for optimizing delay and energy consumption based on federated learning according to claim 7, characterized in that: The total energy consumption to complete a model aggregation is expressed as: In the above formula, E represents the total energy consumption to complete a model aggregation. represents the energy consumption of extracting feature data, Indicates the energy consumption of uploading characteristic data, represents the energy consumption required for convolutional neural network training, Represents the energy consumption required for parameter aggregation.

9. A method for optimizing delay and energy consumption based on federated learning as claimed in claim 6 or 8, characterized in that: The total delay energy consumption to complete a model aggregation is expressed as: min P=γ E E+c T T In the above formula, P is the total delay energy consumption to complete a model aggregation, E is the total energy consumption to complete a model aggregation, T is the total delay to complete a model aggregation, γ E and γ T is the normalized parameter of aggregation delay and energy consumption, which is and 10. A method for optimizing delay and energy consumption based on federated learning as claimed in claim 9, characterized in that: The total delay energy consumption of completing a model aggregation is optimized using a particle swarm algorithm. Each particle in the group updates its position and speed to find the optimal solution in the solution space. The particle position update is expressed as: The particle velocity update is expressed as: In the above formula, denote the d-th dimension component of the position vector of particle i in the k-th and k+1-th iterations respectively, denote the d-th dimension component of the velocity vector of particle i in the k-th and k+1-th iterations, respectively; c1 and c2 are the individual and group learning factors, respectively; w is the inertia weight, represents the local optimum, represents the global optimal value, r1 and r2 are random numbers.

Images