A federated learning communication method for a cloud radio access network

Abstract

The application discloses a kind of federal learning communication methods for cloud radio access network, its characteristics are the method includes: S1, constructs the federal learning framework for cloud radio access network, the federal learning system of cloud radio access network and quantization neural network, training, cloud radio access network communication model;S2, convergence analysis is carried out to federal learning framework;S3, construct federal learning system energy model, and carry out convergence analysis, according to the convergence result and energy consumption model constructs an optimization problem of system energy minimization, and solve the problem by alternating optimization joint optimization.This application compared with prior art has by quantization neural network using low bit data reduces the energy consumption when training, by using the characteristics of high scalability, throughput and coverage of cloud radio access network, realize the federal learning framework of low energy consumption, solve the problem of large energy consumption of wireless federal learning.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and in particular to a federated learning communication method for cloud wireless access networks. Background Technology

[0002] Federated learning is a distributed machine learning model that uses device data from various devices for training and inference. It is now widely used in recommendation systems, medical diagnosis, and other scenarios. Its main advantage lies in avoiding the leakage of device privacy data caused by the transmission of raw device data, effectively protecting device privacy. Federated learning uses local data for model training. A centralized federated learning system typically consists of a central server and multiple local devices. The training steps are roughly as follows: First, each device trains on its local data to obtain local model parameters, which are then uploaded to the central server. The central server aggregates the data received from each local device to obtain a new round of global model parameters. Then, the central server distributes the new round of global model parameters to each device, and so on until the model converges. Centralized federated learning, through this learning and update method, effectively protects device privacy while achieving a relatively good final model performance. However, with the rapid development of deep learning and machine learning, more complex models have become mainstream. Their primary characteristic is an exponential increase in the number of model parameters; for example, neural network models may have hundreds of thousands, millions, or even more model parameters. In wireless communication scenarios, due to the characteristics of wireless channel interference and the limited resources of various devices and channels, such large-scale data transmission can lead to problems such as transmission errors and transmission delays, which can adversely affect the training of federated learning models and are difficult to implement in real-world systems.

[0003] Most existing federated learning technologies use convolutional neural networks for training. These neural networks, with their large number of parameters, typically rely on large model numbers and complex computations to handle tasks. This is unacceptable for devices with limited energy resources. It is difficult to avoid and eliminate storage and computational redundancy in convolutional neural networks, leading to communication bottlenecks such as large communication throughput, communication latency, and high energy consumption. Summary of the Invention

[0004] The purpose of this invention is to provide a green federated learning communication method for cloud wireless access networks (WLANs) to address the shortcomings of existing technologies. This method employs a cloud WLAN and quantized neural network approach. The quantized neural network quantizes data during transmission, and low-bit data sparsity is used to reduce the amount of data transmitted. This enables model training and transmission during the training process, effectively reducing energy consumption during training and transmission. Leveraging the high scalability, throughput, and coverage of cloud WLANs, a low-energy-consumption federated learning framework is achieved. This effectively solves the problem of high energy consumption in most current wireless federated learning methods, thereby alleviating communication bottlenecks and enabling better development of federated learning applications in wireless communication scenarios. It has promising application prospects and commercial value.

[0005] The objective of this invention is achieved as follows: a federated learning communication method for cloud wireless access networks, characterized by the use of quantized neural networks to quantize data during transmission to achieve model training and transmission during the training process, which can effectively reduce the energy consumption of training and transmission. Combined with cloud wireless access network technology, it can improve resource utilization and reduce energy consumption, thereby improving network performance.

[0006] This invention addresses the problem of a large number of model parameters by designing a quantization operation for the parameter information to be transmitted. Quantization maps continuously variable values ​​to multiple discrete values, effectively reducing transmission volume. The goal is to minimize the number of bits required for transmission while retaining as much information as possible from the original data summary, minimizing data error. Quantization effectively reduces communication transmission volume, thus alleviating communication latency and bottlenecks. Existing work and designs exist for a green and efficient federated learning communication method for practical application. These works primarily focus on designing the number of quantization bits during model training and transmission, or using quantization or sparsity reduction to reduce the amount of data transmitted and alleviate communication bottlenecks due to the large number of model parameters. However, these works do not consider federated learning in cloud wireless access network scenarios. Cloud wireless access networks refer to a distributed approach that extends base stations remotely and centralizes all or part of the baseband processing resources to form a baseband resource pool for unified management and dynamic allocation. This effectively reduces energy consumption, improves resource utilization, and enhances access network performance. Therefore, this invention combines federated learning and cloud radio access networks to design a federated learning communication method for cloud radio access networks, specifically including the following steps:

[0007] S1. Construction of the Federated Learning Framework

[0008] First, a federated learning system consisting of a central server, wireless devices, wireless channels, and remote radio heads is modeled, and the model training task is modeled. Then, an algorithm is developed to complete the model training task. Next, the communication model between the server, remote radio heads, and devices is modeled. Finally, the three are integrated to obtain a complete federated learning framework. Based on the proposed federated learning framework, a system energy model is constructed.

[0009] S2. Based on the federated learning framework proposed in step S1, perform convergence analysis.

[0010] S3. Based on the convergence analysis results and the constructed system energy model, construct an optimization problem to minimize system energy consumption, and solve the optimization problem of minimizing system energy consumption by alternating joint optimization, so as to reduce system energy consumption while ensuring model convergence.

[0011] Step S1 specifically includes:

[0012] S11. Establish a model for a wireless federated learning system.

[0013] Treating federated learning conducted off-grid via cloud wireless access network as One single-antenna device, one single-antenna central server. Each single-antenna remote RF head jointly completes the model training process. Each remote RF head is connected to the central server via a forward link with limited capacity. It is assumed that each single-antenna device has a local dataset. ,in For the size of the dataset for a single antenna device, This represents the input vectors and corresponding output results of the dataset. The dataset for the entire system can be represented as... The number of datasets is .

[0014] The task of training the model is modeled as a loss function. For each single-antenna device, its local loss function can be constructed as follows: ,in For equipment For data samples The parameters are loss function, For local datasets The goal of federated learning systems is to learn a shared model by minimizing the sum of the local loss functions of the edge devices. Based on the local loss function, the global loss function optimized by the central server can be modeled as follows: .

[0015] S12. Constructing algorithms for model training tasks

[0016] The algorithm used for model training can be described as an iterative training process, the first... The specific process of round iteration is described in the following steps:

[0017] 1) Equipment Selection: The central server determines the set of single-antenna devices to participate in this iteration and denotes them as follows: This system assumes that the number of single-antenna devices selected in each round is a constant. ;

[0018] 2) Broadcasting the global model: The central server broadcasts the global model for the current round. For all single-antenna devices;

[0019] 3) Local model update: When a single-antenna device receives the global model broadcast for the current round. Subsequently, each single-antenna device trains its local parameters using a quantized neural network. The quantization method employed by each single-antenna device is random quantization, as described in equation (c) below:

[0020] (c).

[0021] in, It is a component in any model parameter; Represents logarithmic values The sign-taking operation: when If it is a positive number, take ;when If it is negative, take ;when for ,Pick . The expression is in This represents the minimum value in the parameter vector. This represents the maximum value in the parameter vector. Indicates using The number of all quantized numbers that can be represented by a bit. Here we use... To represent uniform distribution The quantization value in the above quantization scheme is applied to the first quantization step of the neural network of a single-antenna device. On the upper layer, the quantized first layer can be obtained. The parameters of the layer are , then the first The output of the layer is ,in For the first The activation function of the layer, For the first The output result after layer quantization. Next, the output result after layer quantization. The output of the layer is quantized using the quantization method described above, and the quantized result is applied to the neural network to calculate the first... The layer's output. Then, the model parameters are truncated. ,in The function represents the Each greater than The component is truncated as less than The component is truncated as If the range is within arrive The numbers between these values ​​are preserved. Each single-antenna device uses the quantization neural network described above for... Local training , This is the operation of calculating the gradient based on the local loss function. Indicates the first Round of global training Local training equipment Model parameters, Indicates the first The learning rate during global training. Indicates the first During global training, the device The data used. Therefore, the model difference that the current device needs to transmit is: ,in For local training Model parameters after wheel rotation To reduce data transmission volume in digital communication, the previously mentioned random quantization method is used to quantize the model difference to be transmitted, resulting in a quantized device. The model difference is .

[0022] 4) Model aggregation and global model update: The single-antenna devices participating in this round of training will aggregate the quantized model differences. The data is transmitted to the central server, which then aggregates and averages the data before updating the global model parameters as follows: ,in For the first The system consists of a set of devices. After the central server updates the global model parameters, it distributes the updated global model parameters to each device, and then the devices perform local multi-round training.

[0023] S13. Establish the cloud wireless access network communication model between the central server and devices during the model aggregation process in step S12.

[0024] No. During round-based global training, each single-antenna device will quantize the model difference. Send to One remote radio head. Assume that the subcarriers allocated to each single-antenna device are fixed, and the set of subcarriers allocated to a single-antenna device is... Assuming that any given subcarrier will be allocated to only one device, let the device... to remote radio frequency head In subcarrier The channel coefficients between are ,equipment In subcarrier The transmission power on is , For remote radio frequency head In subcarrier noise and ,in For noise variance, then in subcarrier Above, remote radio frequency head The signal received from a single-antenna device can be represented as ,in It is an orthogonal frequency division multiplexing symbol that contains the quantized model difference. Information about the components. Additionally, it is assumed here... It is independent of and of.

[0025] Each remote radio head The received signals are sent to the central server via forward pass-back. Each remote RF head First, the baseband transmission signal is quantized, and then the quantized signal is transmitted to the central server. Indicates remote radio frequency head In subcarrier Quantization noise on ,in The noise variance can be expressed as Therefore, the frequency domain baseband signal received at the subcarrier is: Assuming and Independent, each remote RF head The KQ modulation method is used; assuming the remote RF head... use When a bit is quantized, then... Quantization level. This can be obtained from the remote RF head. The transmission rate to the central server is: ,in This is the system's maximum transmission rate. For remote radio frequency head In subcarrier The transmission rate on the subcarrier can be obtained from the single-antenna device to the central server. The transmission rate on the network is expressed by the following equation (l):

[0026] (l).

[0027] So, from the equipment The transmission rate to the central server can be expressed as the transmission rate across all devices. The sum of the transmission rates on the subcarriers used, .

[0028] S14. Constructing a federated learning energy model for cloud wireless access networks.

[0029] 1) Device computational energy consumption model: This federated learning system uses a typical two-dimensional processing chip, which consists of a parallel neuron array, It consists of a multiplication-accumulation unit, a main buffer, and a local buffer. The weights and activations of the current layer are stored in the main buffer, while the used weights and activations are cached in the local buffer. Let... At the highest precision level, the energy consumed by the multiplication-accumulation operation can be expressed as: ,in , Let be a constant representing energy. The energy consumption required to access the local buffer can be expressed as... The energy required to access the main buffer is expressed as... Therefore, for a single-antenna device in the first... The energy consumed in a single round of local training during round-robin sessions is expressed as: The energy consumed in batch processing, activation function calculation, and bias calculation can be expressed as: .in, This represents the number of intermediate outputs in the entire neural network. The energy consumed by accessing weights from the buffer can be expressed as: .in, This represents the number of weights. The energy consumed in obtaining the activation function from the buffer can be expressed as: .

[0030] 2) Device Transmission Energy Consumption Model: Considering transmission delay, device transmission power, and fronthaul capacity, this invention proposes an energy consumption model for cloud wireless access network communication based on orthogonal frequency division multiple access. First, the entire uplink transmission delay is divided into two parts: the delay caused by transmitting local training updates from a single-antenna device to a remote radio head via the wireless channel, and the delay caused by transmitting from the remote radio head to the central server via the frontlink. The uplink delay from the single-antenna device to the remote radio head via the wireless channel is given by the following equation (u):

[0031] (u).

[0032] in, For users transmission rate For users In subcarrier The transmission rate on the device.

[0033] From the remote RF head via the forward link The forwarding delay caused by the forwarded signal to the central server is given by the following equation (v):

[0034] (v).

[0035] in, This indicates the number of bits used to quantize the real and imaginary parts of the uplink signal.

[0036] Similarly, the total power consumption is divided into two parts: the energy consumed by a single antenna device during transmission and the power consumed by the forward link. Let... This indicates the transmission power consumption of a single-antenna device. Indicates the first The power consumed by the remote radio frequency head. Wheel, total communication energy consumption of the entire system The following is given .in, For the first The power consumed by each remote radio frequency head Power expressed in relation to rate (in watts per bit per second).

[0037] Step S2 specifically includes:

[0038] First, we provide a notation to aid the convergence proof; specifically, we will... Represented as the number of iterations in a local iteration, Represented as a single-antenna device in iteration Model parameters at that time. Assumptions ,in Each individual antenna device transmits its local model update to the central server, which then performs global aggregation. If... If this happens, the global aggregation step will not occur. Local model updates and global model updates can be defined as follows: ;if Then there is and ;if , and Among them, model difference , For the latest global model broadcast from the central server, It is an auxiliary variable.

[0039] Subsequently, three dummy variables are given for the convergence analysis: , , Define the gradient vector and its auxiliary variables: , Based on the properties of auxiliary variables, we can obtain... , Based on the quantization and update methods of federated learning models, the properties of iteration are deduced as follows: Due to the launch Given a learning rate , It can be deduced ,in .

[0040] Step S3 specifically includes:

[0041] S31. Optimization Problem Construction

[0042] To minimize the energy consumed by the entire system, the number of quantization bits in the quantization neural network is quantized. Forward link bits and equipment transmission power The optimization problem is expressed by the following equation (y):

[0043] .

[0044] in, This represents the energy consumed by a single-antenna device to perform a single round of local update calculations.

[0045] ,in This indicates the calculation of energy consumption. This represents the energy consumption of retrieving neural network weights from the cache. This indicates the energy consumption for retrieving activation parameters from the cache. Indicates the first Wheel of all in the device set Transmission power consumption of devices in the process: ,in Indicates from device The time consumed in transmitting data to the remote RF head. Indicates equipment Transmission power, This indicates the time taken to transmit data from the remote radio head to the central server. Indicates remote radio frequency head Number of bits used for quantization This refers to the power transmitted back from the remote radio head.

[0046] One constraint in the optimization problem is the number of quantization bits in the quantization neural network. The first constraint is an integer constraint, and the second constraint is a constraint on the system training accuracy. Let be the error constant, constraint three be the power constraint of the device, constraint four be the constraint on the number of forward link bits, and constraint five be the integer constraint on the number of forward link bits. Clearly, the above problem is a non-convex optimization problem; therefore, this invention uses an alternating optimization approach to solve this problem.

[0047] S32. A joint optimization method for device power, forward link bits, and quantized neural network bits.

[0048] Power transmitted through fixed equipment and number of bits in the forward link Two variables, solving for the number of quantization bits in a quantized neural network. Through convergence analysis, the above optimization problem can be rewritten as the optimization problem expressed by equation (i) below:

[0049]

[0050] (i).

[0051] in, , , , , , For system constants, This refers to the number of devices participating in each round of training in the actual system. Because the problem is within the scope It is differentiable, therefore it can be solved using a linear search algorithm. Solve the problem.

[0052] Number of quantization bits in a fixed-quantization neural network and the number of forward link bits Solve for the transmission power of fixed equipment Through convergence analysis, the optimization problem in equation (i) above can be rewritten as the optimization problem expressed in equation (ii) below:

[0053]

[0054] (ii).

[0055] The above problem can be transformed into a fractional programming problem.

[0056] By designing auxiliary variables ,make , Due to Cauchy's inequality, the original minimization optimization problem is transformed into a maximization optimization problem. The new fractional programming problem can be expressed by the following equation (iii):

[0057]

[0058] (iii).

[0059] It can be seen that the new fractional programming problem is a conventional fractional programming problem, which can be solved using the methods for solving conventional fractional programming problems. First, we design auxiliary variables. The above problem can be rewritten as equation (iv):

[0060]

[0061] (iv)

[0062] Observed right It is a concave function. right It is a convex function, and we use an alternating optimization method to solve it: when fixed , The optimal solution is When fixed Yes, the problem becomes a convex problem, so it can be solved using the CVX optimization package.

[0063] Number of quantization bits in a fixed-quantization neural network and the transmission power of fixed equipment Solve for the number of bits in the forward link. First, ignore the integer constraint. Design auxiliary variables The original optimization problem can be written as the following optimization problem (v):

[0064]

[0065] (v).

[0066] Because, objective function right It is non-convex; here, a first-order approximation method is used to approximate the non-convex objective function. This can be transformed into the following equation (vi) to represent a convex objective function:

[0067] (vi).

[0068] in, This indicates the energy consumed by the device to send signals. This represents the energy consumed by the remote radio head in transmitting signals. Similarly, constraints will be applied. Transform it into a convex constraint using a first-order approximation method: ,in It is an auxiliary variable.

[0069] The original problem is transformed into a convex problem through approximation, and convex optimization can be used to solve it. The results of alternating optimizations of device power, forward link rate allocation, and the number of bits in the quantized neural network are used for the next round of optimization until the final result converges, which is the optimized system result.

[0070] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:

[0071] 1) This invention adopts a quantized neural network and proposes a green and efficient federated learning method in combination with cloud wireless access network.

[0072] 2) By employing a quantized neural network during device training, this invention effectively solves the problem of model redundancy during training, greatly reducing the computational load on the device. In terms of transmission, this invention combines cloud wireless access network, thereby improving network resource utilization and enhancing access network performance.

[0073] 3) This invention addresses the optimization problems of the number of bits in the quantized neural network, the number of forward and backhaul bits in the cloud wireless access network, and the power consumption of the device. It proposes an alternating optimization method, which greatly reduces the communication and computational overhead of the device in federated learning, improves the communication efficiency of wireless federated learning, and reduces the overall system energy consumption. Attached Figure Description

[0074] Figure 1 is a system diagram of the architecture of this invention;

[0075] Figure 2 is a comparison of system energy consumption between the embodiment and other optimization methods under the same model accuracy settings on the MNKST dataset.

[0076] Figure 3 This is a comparison chart showing whether optimizing device transmission power has an impact on the final system energy consumption compared to other optimization methods.

[0077] Figure 4 To investigate the relationship between total system energy consumption and the number of quantization bits in the quantization neural network under varying training target accuracy of the federated learning model. Detailed Implementation

[0078] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0079] Example 1

[0080] See Figure 1 According to the architecture of this invention, efficient communication based on federated learning in a cloud wireless access network environment specifically includes the following steps:

[0081] S1. Construction of the Federated Learning Framework

[0082] First, a federated learning system consisting of a central server, wireless devices, wireless channels, and remote radio heads is modeled, and the model training task is modeled. Then, an algorithm is developed to complete the model training task. Next, the communication model between the server, remote radio heads, and devices is modeled. Finally, the three are integrated to obtain a complete federated learning framework. Based on the proposed federated learning framework, a system energy model is constructed.

[0083] S2. Based on the federated learning framework proposed in step S1, perform convergence analysis.

[0084] S3. Based on the convergence analysis results and the constructed system energy model, construct an optimization problem to minimize system energy consumption, and solve the optimization problem of minimizing system energy consumption by alternating joint optimization, so as to reduce system energy consumption while ensuring model convergence.

[0085] Step S1 specifically includes:

[0086] S11. Establish a model for a wireless federated learning system.

[0087] Treating federated learning conducted off-grid via cloud wireless access network as Single-antenna device, one single-antenna central server, Each device employs a single-antenna remote RF head to collaboratively complete the model training process. Each remote RF head is connected to the central server via a forward link with limited capacity. It is assumed that each device has a local dataset. ,in For the size of the device dataset, This represents the input vectors and corresponding output results of the dataset. The dataset for the entire system can be represented as... The number of datasets is .

[0088] The task of training the model is modeled as a loss function. For each device... For example, its local loss function can be constructed as follows: ,in For equipment For data samples The parameters are loss function, For local datasets The goal of federated learning systems is to learn a shared model by minimizing the sum of the local loss functions of the edge devices. Based on the local loss function, the global loss function optimized by the central server can be modeled as follows: .

[0089] S12. Constructing algorithms for model training tasks

[0090] The algorithm used for model training can be described as an iterative training process, the first... The specific process of round iteration is described in the following steps:

[0091] 1) Equipment Selection: The central server determines the set of devices participating in this iteration and denotes it as... This system assumes that the number of devices selected in each round is a constant. .

[0092] 2) Broadcasting the global model: The central server broadcasts the global model for the current round. Give it to all devices.

[0093] 3) Local model update: on the device Receive the global model broadcast for the current round After that, each device The system is trained on local parameters, using a quantized neural network. Each device... The quantization method used is random quantization, which is described in equation (c) below:

[0094] (c).

[0095] in, It is one of the components in any model parameter. Represents logarithmic values The sign-taking operation: when If it is a positive number, take ;when If it is negative, take ;when for ,Pick . The expression is in This represents the minimum value in the parameter vector. This represents the maximum value in the parameter vector. Indicates using The number of all quantized numbers that can be represented by a bit. Here we use... To represent uniform distribution The quantization value in the image. The above quantization scheme is then applied to the device. The first neural network On the upper layer, the quantized first layer can be obtained. The parameters of the layer are , then the first The output of the layer is ,in For the first The activation function of the layer, For the first The output result after layer quantization. Next, the output result after layer quantization. The output of the layer is quantized using the quantization method described above, and the quantized result is applied to the neural network to calculate the first... The layer's output. Then, the model parameters are truncated. ,in The function represents the Each greater than The component is truncated as less than The component is truncated as If the range is within arrive Numbers between these values ​​are retained as their original values. Each device... Using the quantization neural network described above Local training , This is the operation of calculating the gradient based on the local loss function. Indicates the first Round of global training Local training equipment Model parameters, Indicates the first The learning rate during global training. Indicates the first During global training, the device The data used. Therefore, the model difference that the current device needs to transmit is: ,in For local training Model parameters after wheel rotation To reduce data transmission volume in digital communication, the previously mentioned random quantization method is used to quantize the model difference to be transmitted, resulting in a quantized device. The model difference is .

[0096] 4) Model aggregation and global model update: Devices participating in this round of training The quantized model difference The data is transmitted to the central server, which then aggregates and averages the data before updating the global model parameters as follows: ,in For the first The system consists of a set of devices. After the central server updates the global model parameters, it distributes the updated global model parameters to each device, and then the devices perform local multi-round training.

[0097] S13. Establish the cloud wireless access network communication model between the central server and devices during the model aggregation process in S12.

[0098] No. During global training, each device The quantized model difference Send to Each device has a remote radio frequency head. Assume each device... The allocated subcarriers are fixed and assigned to the devices. The set of subcarriers is This assumes that any subcarrier will be allocated to only one device. Let the device... to remote radio frequency head In subcarrier The channel coefficients between are ,equipment In subcarrier The transmission power on is , For remote radio frequency head In subcarrier noise and ,in For noise variance, then in subcarrier Above, remote radio frequency head Received device The transmitted signal can be represented as ,in It is an orthogonal frequency division multiplexing symbol that contains the quantized model difference. Information about the components. Additionally, it is assumed here... It is independent of and of.

[0099] Each remote radio head The received signals are sent to the central server via forward pass-back. Each remote RF head First, the baseband transmission signal is quantized, and then the quantized signal is transmitted to the central server. Indicates remote radio frequency head In subcarrier Quantization noise on ,in The noise variance can be expressed as Therefore, the frequency domain baseband signal received at the subcarrier is... Here we assume and Independent. Each remote RF head The KQ modulation method is used, assuming the remote RF head... use When a bit is scalar quantized, then... Quantization level. This can be obtained from the remote RF head. The transmission rate to the central server is ,in This is the system's maximum transmission rate. For remote radio frequency head In subcarrier The transmission rate on the device To the central server on subcarrier The transmission rate on the network is expressed by the following equation (l):

[0100] (l).

[0101] So, from the equipment The transmission rate to the central server can be expressed as the transmission rate across all devices. The sum of the transmission rates on the subcarriers used, .

[0102] S14. Constructing a federated learning energy model for cloud wireless access networks.

[0103] 1) Equipment Energy Consumption Model: This system uses a typical two-dimensional processing chip. This chip consists of a parallel neuron array, It consists of a multiplication-accumulation unit, a main buffer, and a local buffer. The weights and activations of the current layer are stored in the main buffer, while the used weights and activations are cached in the local buffer. Let... At the highest precision level, the energy consumed by the multiplication-accumulation operation can be expressed as: ,in , Let be a constant representing energy. The energy consumption required to access the local buffer can be expressed as... The energy required to access the main buffer is expressed as... Therefore, for equipment In the The energy consumed in a local round of round-based local training is expressed as follows: The energy consumed in batch processing, activation function calculation, and bias calculation can be expressed as: ,in This represents the number of intermediate outputs in the entire neural network. The energy consumed by accessing weights from the buffer can be expressed as... ,in This represents the number of weights. The energy consumed in obtaining the activation function from the buffer can be expressed as... .

[0104] 2) Device Transmission Energy Consumption Model: Considering transmission delay, device transmission power, and fronthaul capacity, an energy consumption model for cloud wireless access network communication based on orthogonal frequency division multiple access (OFDM) is proposed. First, the entire uplink transmission delay is divided into two parts: the delay caused by sending local training updates from the device to the remote radio head via the wireless channel, and the delay caused by transmitting updates from the remote radio head to the central server via the frontlink. The energy consumption model for cloud wireless access network communication based on orthogonal frequency division multiple access (OFDM) is as follows: ... The upload delay to the remote RF head is given by the following formula: ,in From the remote RF head via the forward link The forwarding delay caused by the forwarded signal to the central server is given by the following formula: ,in This represents the number of bits used to quantize the real and imaginary parts of the uplink signal. Similarly, total power consumption is divided into two parts: the energy consumed by the device during transmission and the power consumed by the forward link. Let... Indicates equipment Transmission power consumption, Indicates the first The power consumed by the remote radio frequency head. Wheel, total communication energy consumption of the entire system The following is given ,in For the first The power consumed by each remote radio frequency head Power expressed in relation to rate (in watts per bit per second).

[0105] Step S2 specifically includes:

[0106] First, we provide a notational representation to aid in the convergence proof. Specifically, we will... Represented as the number of iterations in a local iteration, Represented as device In iteration Model parameters at that time. Assumptions ,in Each device transmits its local model updates to the central server, which then performs global aggregation. If In this case, the global aggregation step will not occur. Local model updates and global model updates can be defined as follows: ;if Then there is and ;if , and Among them, model difference , For the latest global model broadcast from the central server, It is an auxiliary variable.

[0107] Subsequently, three dummy variables are given for the convergence analysis: , , Define the gradient vector and its auxiliary variables: , Based on the properties of auxiliary variables, we can obtain... , Based on the quantization and update methods of federated learning models, the properties of iteration are deduced as follows: Due to the launch Given a learning rate , It can be deduced ,in .

[0108] Step S3 specifically includes:

[0109] S31. Optimization Problem Construction

[0110] To minimize the energy consumed by the entire system, the number of quantization bits in the quantization neural network is quantized. Forward link bits and equipment transmission power The optimization problem is expressed by the following equation (y):

[0111]

[0112]

[0113]

[0114]

[0115] (y).

[0116] in, Indicates equipment Energy consumed in a single round of computation for local updates: ,in This indicates the calculation of energy consumption. This represents the energy consumption of retrieving neural network weights from the cache. This represents the energy consumption of retrieving activation parameters from the cache. Indicates the first Wheel of all in the device set Transmission power consumption of devices in the process: ,in Indicates from device The time consumed in transmitting data to the remote RF head. Indicates equipment Transmission power, This indicates the time taken to transmit data from the remote radio head to the central server. Indicates remote radio frequency head Number of bits used for quantization This refers to the power transmitted back from the remote RF head. One constraint in the optimization problem is the number of quantization bits in the quantization neural network. The first constraint is an integer constraint, and the second constraint is a constraint on the system training accuracy. Let be the error constant, constraint three be the power constraint of the device, constraint four be the constraint on the number of forward link bits, and constraint five be the integer constraint on the number of forward link bits. Clearly, the above problem is a non-convex optimization problem; therefore, this invention uses an alternating optimization approach to solve this problem.

[0117] S32. Joint optimization of device power, forward link bits, and quantization neural network bits.

[0118] Power transmitted through fixed equipment and number of bits in the forward link Two variables, solving for the number of quantization bits in a quantized neural network. Through convergence analysis, the above optimization problem can be rewritten as the optimization problem in equation (i) below:

[0119]

[0120] (i).

[0121] in, , , , , , For system constants, This refers to the number of devices participating in each round of training in the actual system. Because the problem is within the scope It is differentiable, therefore it can be solved using a linear search algorithm. Solve the problem.

[0122] Number of quantization bits in a fixed-quantization neural network and the number of forward link bits Solve for the transmission power of fixed equipment Through convergence analysis, the optimization problem in equation (i) above can be rewritten as the optimization problem in equation (ii) below:

[0123]

[0124] (ii).

[0125] The above problem can be transformed into a fractional programming problem, which can be solved by designing auxiliary variables. ,make , Due to Cauchy's inequality, the original minimization optimization problem is transformed into a maximization optimization problem. The new fractional programming problem can be expressed as follows (iii):

[0126]

[0127] (iii).

[0128] It can be seen that the new fractional programming problem is a conventional fractional programming problem, which can be solved using the methods for solving conventional fractional programming problems. First, we design auxiliary variables. The above problem can be rewritten as equation (iv):

[0129]

[0130] (iv).

[0131] Observed right It is a concave function. right It is a convex function, and we use an alternating optimization method to solve it: when fixed , The optimal solution is When fixed Yes, the problem becomes a convex problem, so it can be solved using the CVX optimization package.

[0132] Number of quantization bits in a fixed-quantization neural network and the transmission power of fixed equipment Solve for the number of bits in the forward link. First, ignore the integer constraint. Design auxiliary variables The original optimization problem can be written as the optimization problem expressed by the following equation (v):

[0133]

[0134] (v).

[0135] Due to the objective function right It is non-convex; here, a first-order approximation method is used to approximate the non-convex objective function. This can be transformed into a convex objective function represented by the following equation (vi):

[0136] (vi).

[0137] in, This indicates the energy consumed by the device to send signals. This represents the energy consumed by the remote radio head in transmitting signals. Similarly, constraints will be applied. Transform it into a convex constraint using a first-order approximation method: ,in It is an auxiliary variable. In this way, the original problem is transformed into a convex problem through approximation, and convex optimization methods can be used to solve the problem.

[0138] The results of alternating optimization of device power, forward link rate allocation, and quantization neural network bit count are used for the next round of optimization until the final result converges, which is the system optimization result.

[0139] See Figure 2 To verify the effectiveness of the present invention, the above embodiments are compared with other optimization methods on the MNIST dataset to evaluate the final system energy consumption. Figure 2As shown, lower total energy consumption indicates better optimization performance, and the effect of this invention is significantly better than that of other methods. Figure 2 As can be seen, compared with other optimization schemes (only optimizing device transmission power, or only optimizing the number of quantization neural network bits, or only optimizing the number of forward link bits, or only optimizing the number of forward link bits and the number of quantization neural network bits), the method proposed in this invention can effectively reduce system energy consumption, improve the energy consumption of federated learning and the energy consumption of transmission signals, and realize efficient and green digital federated learning.

[0140] See Figure 3 By comparing the two schemes of optimizing equipment power and distributing equipment power equally, it can be seen that optimizing equipment power can significantly reduce the energy consumption of the system.

[0141] See Figure 4 The relationship between total energy consumption and optimal accuracy level is illustrated by changing the target accuracy. The graph shows that the optimal accuracy level initially decreases as the target accuracy increases, indicating a trade-off between target accuracy and the optimal number of quantized bits in the federated learning model's performance. This invention utilizes a quantized neural network combined with a cloud wireless access network to significantly reduce the energy consumption of the system during federated learning training, effectively improving the system's energy efficiency.

[0142] The above embodiments are merely illustrative of the present invention and are not intended to limit the scope of the present invention. All equivalent implementations of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A federated learning communication method for cloud wireless access networks, characterized in that, This method employs a quantized neural network and utilizes a cloud wireless access network to achieve low-energy federated learning. The specific steps include: S1. Construct the federated learning framework according to the following steps: S1-1: Model the federated learning system consisting of a central server, wireless devices, wireless channels, and remote radio heads, and complete the model training task. Its input is the system architecture parameters and the local dataset, and its output is the mathematical objective function F(w) used to describe the training objective of federated learning. S1-2: Constructing algorithms for model training tasks; S1-3: Model the communication model between the server, remote RF head and the device. Its input is wireless channel parameters, power and quantization configuration. The output is the mathematical model of the complete digital communication link from the device to the central server under the entire C-RAN architecture. The core output is the achievable transmission rate R_k of each device. S1-4: Integrate the models and algorithms completed in the above three steps to obtain a complete federated learning framework; S1-5: Construct a system energy model based on the federated learning framework. Its inputs integrate computing hardware characteristics, neural network structure, and communication model parameters, and its output is a quantitative mathematical model of the total energy consumption of the system. S2. Perform convergence analysis on the federated learning framework obtained in steps S1-4. S3. Based on the convergence analysis results, the system energy model is optimized to minimize system energy consumption through alternating joint optimization. This reduces system energy consumption while ensuring model convergence, thus minimizing system energy consumption. The alternating joint optimization method involves alternating optimization of device power, forward link bits, and quantized neural network bits, and then performing the next round of optimization until the final result converges, which is the system optimization result.

2. The green federated learning communication method for cloud wireless access networks according to claim 1, characterized in that, Step S1 specifically includes: S11. Constructing a model for a wireless federated learning system Treating federated learning conducted off-grid via cloud wireless access network as A single-antenna device, a single-antenna central server, and The model training process is jointly completed by several single-antenna remote radio heads, with each remote radio head connected to the central server via a forward link with limited capacity; it is assumed that each single-antenna device has a local dataset. ,in For the size of the dataset for a single antenna device, The dataset represents the input vectors and corresponding outputs of the dataset. The dataset for the entire federated learning system is represented as follows: The number of datasets is ; The task of training the model is modeled as a loss function. For each single-antenna device, its local loss function is constructed by the following equation (a): （a）； in, For single-antenna devices, data samples The loss function; For local datasets The trained local model; The goal of the federated learning system is to learn a shared model by minimizing the sum of the local loss functions of the edge devices. Based on the local loss function, the global loss function optimized by the central server is modeled by the following equation (b): （b）； S12. Constructing algorithms for model training tasks The algorithm used for model training is an iterative training process, the first of which... The round of iterations is as follows: 1) Equipment Selection: The central server determines the set of devices participating in this iteration and denotes it as... This system assumes that the number of single-antenna devices selected in each round is a constant. ; 2) Broadcasting the global model: The central server broadcasts the global model for the current round. For all single-antenna devices; 3) Local model update: When a single-antenna device receives the global model broadcast for the current round. After that, each single-antenna device The local parameters are trained using a quantized neural network. Each single-antenna device employs random quantization, which is described by the following equation (c): （c）； in, For parameters The quantified result; It is a component in any model parameter; Represents logarithmic values The sign-taking operation: when If it is a positive number, take ;when If it is negative, take ;when for ,Pick ; The quantification parameter is expressed by the following equation (d): （d）； in, This represents the minimum value in the parameter vector; This represents the maximum value in the parameter vector; Indicates using The number of all quantized digits represented by bits; use Indicates uniform distribution in The quantized value in the data is applied to the first step of the neural network of a single-antenna device. On the layer, we obtain the quantized first... The parameters of the layer are: , then the first The output of the layer is ,in For the first Activation function of the layer; For the first The output result after layer quantization; the first layer... The output of the layer is quantized in the manner described above, and the quantized result is applied to the neural network to calculate the first... The layer's output; subsequently, the model parameters are truncated. ,in The function represents the Each greater than The component is truncated as less than The component is truncated as If the range is within arrive The numbers between them are retained in their original form; Each single-antenna device uses the quantization neural network described above, as shown in equation (e) below. Local training rotation: （e）； in, This is the operation of calculating the gradient based on the local loss function; Indicates the first Round of global training The model parameters of the single-antenna device are trained locally. Indicates the first The learning rate for global training rounds; Indicates the first Single-antenna device during global training Data used; The model difference that the current single-antenna device needs to transmit, calculated by equation (f) below, is obtained. : （f）； in, For local training Model parameters after the wheel; ; model difference Quantization is performed to obtain the model difference of the quantized single-antenna device. ; 4) Model aggregation and global model update: The single-antenna devices participating in this round of training will aggregate the quantized model differences. The data is transmitted to the central server, which then aggregates and averages the data before updating the global model parameters as shown in equation (g): （g）； in, For the first A collection of single-antenna devices for wheels; For the equipment in the Parameters for wheel rotation; After the central server updates the global model parameters, it sends the updated global model parameters to each device for local multi-round training. S13. Establish a cloud wireless access network communication model between the central server and devices. No. During round-based global training, each single-antenna device will quantize the model difference. Send to One remote RF head; assuming the subcarriers allocated to each single-antenna device are fixed, the set of subcarriers allocated to a single-antenna device is as follows. Assuming that any subcarrier will only be allocated to a single-antenna device, the single-antenna device will be connected to the remote radio head. In subcarrier The channel coefficients between are Single-antenna devices on subcarriers The transmission power on is , For remote radio frequency head In subcarrier noise and , in For noise variance, then in subcarrier Above, remote radio frequency head Received signal transmitted by single antenna device It can be expressed by the following formula (h): （h）； in, It is an orthogonal frequency division multiplexing symbol that contains the quantized model difference. Information about the components; To be independent and ; Each remote radio head The received signals are sent to the central server via forward pass-back, and each remote RF head... First, the baseband transmission signal is quantized, and then the quantized signal is transmitted to the central server. Indicates remote radio frequency head In subcarrier Quantization noise on ,in The noise variance is expressed by the following equation (i): （i）； So, the frequency domain baseband signal received at the subcarrier It can be expressed by the following equation (j): （j）； in, and Independent quantization noise; Each remote radio head Using KQ modulation, assuming the remote RF head use When a bit is scalar quantized, then... The quantization level is obtained by calculating the value from the remote RF head using the following formula (k). Transmission rate to the central server : （k）； in This represents the system's maximum transmission rate. For remote radio frequency head In subcarrier The transmission rate on the subcarrier is obtained as expressed by the following equation (l): transmission rate : （l）； The transmission rate from a single-antenna device to the central server is expressed by the following formula (m), which is the sum of the transmission rates on all subcarriers used by the single-antenna device: （m）； S14. Constructing a federated learning energy model for cloud wireless access networks. 1) Device computing energy consumption model: A device computing energy consumption model constructed using a two-dimensional processing chip, which consists of a parallel neuron array, It consists of a multiplication-accumulation unit, a main buffer, and a local buffer. The weights and activations of the current layer are stored in the main buffer, while the weights and activations used are cached in the local buffer; let At the highest precision level, the energy consumed by multiplication and accumulation operations It can be expressed by the following formula (n): （n）； in, , A constant expressed in terms of energy; Energy consumed by accessing the local buffer It can be expressed by the following equation (o): （o）； Energy consumed by accessing the main buffer It can be expressed by the following formula (p): （p）； For single-antenna devices in the... Energy consumed in a local round of local training during round-robin It can be expressed by the following formula (q): （q）； Energy consumed in batch processing, activation function and bias calculation It can be expressed by the following formula (r): （r）； in, This represents the number of intermediate outputs in the entire neural network; Energy consumed by accessing weights from the buffer It can be expressed by the following formula (s): （s）； in, Indicates the number of weights; The energy E consumed by the activation function to retrieve the energy from the buffer. A It can be expressed by the following equation (t): ； 2) Device Transmission Energy Consumption Model: An energy consumption model for cloud wireless access network communication based on orthogonal frequency division multiple access is constructed. The entire uplink transmission delay is divided into two parts: the delay caused by sending local training updates from a single-antenna device to a remote radio head via the wireless channel, and the delay caused by transmitting from the remote radio head to the central server via the forward link. The upload delay from the single-antenna device to the remote radio head via the wireless channel is also considered. It can be expressed by the following formula (u): （u）； in, For users transmission rate For users In subcarrier The transmission rate on the device; From the remote RF head via the forward link Forwarding delay caused by forwarding signals to the central server It can be expressed by the following equation (v): （v）； in, This indicates the number of bits used to quantize the real and imaginary parts of the uplink signal; Similarly, the total power consumption consists of two parts: the energy consumed by the device during transmission and the power consumed by the forward link. Let's assume... Indicates equipment Transmission power consumption, Indicates the first The power consumed by each remote radio frequency head; In the The total communication energy consumption of the entire federated learning system. It can be expressed by the following formula (w): （w）； in, For the first The power consumed by each remote radio frequency head This indicates power related to speed.

3. The green federated learning communication method for cloud wireless access networks according to claim 1, characterized in that, Step S2 specifically includes: S21, will This represents the number of iterations in the local iteration. Represented as a single-antenna device in iteration Model parameters at time, assuming ,in Each single-antenna device transmits its local model update to the central server for global aggregation; if Since global aggregation does not occur, the method for updating the local model can be described as gradient descent update as defined in equation (x) below: （x）； in, This is the result after gradient descent; The learning rate; For parameters exist Gradient on the data; For users In the The model parameters are updated globally. if Then there is and ;if , and Among them, model difference ; The latest global model broadcast from the central server; It is an auxiliary variable; S22. Provide three dummy variables for convergence analysis: Define the gradient vector and its auxiliary variables: , Based on the auxiliary variables, we obtain , Based on the quantization and update methods of the federated learning model, the properties of iteration are deduced as follows: ;in , It is a constant. The optimal model parameters are... The variance of the local model parameters. Update the round number for local devices. Let be the upper bound of the infinity norm of the model parameters. For the number of devices, For model error, As an auxiliary variable, For the first Learning rate during rounds; according to and given learning rate ,constant ,roll out ,in ,in For model error, , For the correlation constant, Update the round number for local devices. The optimal model parameters are... Let be the upper bound of the infinity norm of the model parameters. For the process The global model parameters after the global update. As an auxiliary variable, .

4. The green federated learning communication method for cloud wireless access networks according to claim 1, characterized in that, Step S3 specifically includes: S31. Construction of the Optimization Problem Minimize the energy consumed by the entire system and quantize the number of quantization bits in the neural network. Forward link bits and equipment transmission power The optimization problem is represented by the following (y): ； in, This represents the energy consumed by a single-antenna device to perform a single round of local update calculations. ,in Indicates calculated energy consumption; This represents the energy consumption of retrieving neural network weights from the cache. This indicates the energy consumption for retrieving activation parameters from the cache. Indicates the first All single-antenna devices in the round The transmission energy consumption of the equipment in the middle; ,in Indicates from device The time consumed in transmitting data to the remote RF head; This indicates the transmission power of a single-antenna device; This indicates the time taken to transmit data from the remote radio head to the central server; Indicates remote radio frequency head Number of bits used for quantization; This refers to the power transmitted back from the remote radio frequency head. Let be the error constant; The set of subcarriers allocated to a single-antenna device; The transmission rate from the remote radio head m to the central server; S32. Joint optimization of device power, forward link rate allocation, and the number of bits in the quantized neural network. Power transmitted through fixed equipment and number of bits in the forward link Two variables, solving for the number of quantization bits in a quantized neural network. Through convergence analysis, equation (y) can be rewritten as the optimization problem represented by equation (i) below: （i）； in, , , , D and D are system constants, respectively; This refers to the number of devices participating in each round of training in the actual system. ; The maximum number of bits for quantization; Number of quantization bits in a fixed-quantization neural network and the number of forward link bits Solve for the transmission power of fixed equipment Through convergence analysis, equation (i) above can be rewritten as the optimization problem represented by equation (ii) below: (ii); in, The number of devices randomly selected by the server to participate in training in each round; For the number of devices; , , , It is a constant; Update the number of rounds for local devices; For equipment In subcarrier The transmission power; For equipment Maximum transmit power; By designing auxiliary variables ,make , The original minimization optimization problem is transformed into a maximization optimization problem, that is, the new fractional programming problem is expressed by the following equation (iii): (iii); in, and Regarding equipment power The expression; Design auxiliary variables The above equation (iii) can be rewritten as follows (iv): (iv); in, Auxiliary variables introduced; For equipment In subcarrier The transmission power; For equipment Maximum transmit power; Equation (iv) above is solved using an alternating optimization method: when fixed , The optimal solution is When fixed The CVX optimization package can be used to solve this problem. ; Number of quantization bits in a fixed-quantization neural network and the transmission power of fixed equipment Solve for the number of bits in the forward link. Ignore integer constraints Design auxiliary variables The optimization problem can be rewritten from equation (iv) as shown in equation (v): （v）； in, Transmitting power to fixed equipment and the number of quantization bits The energy expression of the device; This represents the maximum number of quantization bits for the remote radio frequency head. To convert the number of quantization bits The introduced auxiliary variables; The non-convex objective function is approximated using a first-order approximation method. The objective function can be transformed into the following equation (vi) to represent a convex function: (we); in, This indicates the energy consumed by the device to send signals; This indicates the energy consumed by the remote radio frequency head to transmit signals; Similarly, constraints Using a first-order approximation, the constraint can be transformed into the following equation (vii) to represent the convex constraint: (vii); in, It is an auxiliary variable; The device power, forward link rate allocation, and number of quantized neural network bits are alternately optimized, and the results are used for the next round of optimization until the final result converges, which is the result of system optimization.

Images