Federated meta-learning system based on air computing and its optimization method and control device

Abstract

The present invention discloses a federated meta-learning system based on air computing, an optimization method and a control device thereof. Each training user device performs meta-learning training locally and calculates local gradients, which helps to provide the generalization ability of the global model and overcome the influence of the local data heterogeneity of the training user device. At the same time, the local gradients of each training user device are sent to the base station on the same time-frequency resources, which helps to improve the spectrum resource utilization of the system. In addition, the preset power constraints of each training user device help to save user energy consumption under the premise of completing the federated meta-learning task based on air computing. Since the transmission power configuration of each training user device and the beamforming vector of the base station are jointly optimized, the federated meta-learning aggregation process based on air computing is globally optimized, the convergence of the federated meta-learning is improved, and a better federated meta-learning training effect is achieved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a federated meta-learning system based on over-the-air computing, an optimization method thereof, a control device, and electronic equipment. Background Art

[0002] In future 6G networks, edge AI offers unlimited possibilities for intelligent connectivity. Federated learning, a representative distributed learning paradigm for edge AI, enables decentralized edge devices to collaborate on training shared models.

[0003] In a federated learning system, each user device uses local data to train a model and then uploads the resulting local gradients to the base station. The base station aggregates the local gradients of each user device to derive a global gradient and uses this gradient to update the global model. Once the update is complete, the base station sends the global model to each training user device. This allows model training to be performed using the local data of each training user device while maintaining data privacy.

[0004] However, in the aforementioned federated learning system, the local data of each user device may be heterogeneous, meaning that the local data between users is not independent and identically distributed. This can affect the performance of federated learning and prevent further improvement. To address this issue, Federated Meta-Learning combines federated learning and meta-learning methods. During local training on each user device, meta-learning is used to calculate gradients twice. The resulting local gradients are then uploaded to the base station for aggregation.

[0005] However, traditional federated meta-learning systems ignore the impact of the wireless channel between user devices and base stations, which limits the application of federated meta-learning systems in wireless communication networks.

[0006] Based on this, there is an urgent need for an improved federated learning method, that is, a method that can improve the generalization performance of the global model in federated learning, overcome the adverse effects of user local data heterogeneity, and at the same time improve the efficiency of wireless communication between user devices and base stations, so that when performing federated learning on user devices, the adverse effects of data heterogeneity can be overcome, thereby improving the training effect of the global model and achieving efficient communication between user devices and base stations. Summary of the Invention

[0007] In response to the shortcomings of the existing technology, the present invention proposes a federated meta-learning system based on over-the-air computing and its optimization method and control device to improve the training performance of traditional federated learning systems.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a federated meta-learning system based on over-the-air computing, which is used to realize federated learning under heterogeneous user data and efficient model transmission and aggregation, wherein the transmitter-receiver structure includes at least one training user device and a base station, each user device uses a power allocation factor to control the transmission power of the local gradient, and the base station configures an aggregated beamforming vector to complete the local gradient aggregation based on over-the-air computing; and adopts the following steps: when entering each training round, the training user device responsible for local training divides its data samples into a training set and a test set; then, the training user device calculates a first gradient based on the samples of the training set, and performs a gradient descent to obtain a local updated parameter; then, the training user device calculates a second gradient based on the samples of the test set based on the local updated parameter, and uploads the second gradient to the base station; the base station aggregates the local gradients of each user device according to the over-the-air computing method, and finally uses the aggregated gradient to update the global model; the training round includes a local training cycle and a communication round, and the uploading process of the local gradients of each training user device is performed on the same time-frequency resources.

[0010] Furthermore, the specific process of the user device performing federated meta-learning training locally is as follows:

[0011] First, the kth training user device sends all its datasets D k Divide into training set and validation set Then based on the training set The first gradient is calculated as follows:

[0012]

[0013] Among them, θ t is the vector of current global model parameters broadcast by the base station to each user at the current t-th training round; When the current t-th training round is reached, the k-th training user device has Upper θ t The gradient obtained by derivation has a total of K tra Training user devices are trained for federated meta-learning, and the labels of each training user device constitute the set K tra ; Is the training set The size of Is the training set The jth training sample; is the loss function l of the kth training user device in the current training roundk On its jth data sample, θ t The gradient obtained by differentiation;

[0014] Next, each training user device performs a gradient descent, which is performed using the following formula:

[0015]

[0016] in, is the localized update parameter for the kth training user device, and α is the local learning rate;

[0017] Then, the kth training user device is trained based on the validation set The second gradient is calculated according to the following formula:

[0018]

[0019] in, is the local gradient to be uploaded to the base station, is the number of users in the kth training set in the current tth training round Upper pair parameters The gradient obtained by derivation is calculated in the same way as the gradient resemblance; For the parameters The first-order gradient of

[0020] The training user device converts the local gradient Perform preprocessing to obtain the local gradient signal vector to be uploaded as follows:

[0021]

[0022] Among them, φ is the preprocessing function, and the base station follows the gradient The dimension d of the local gradient signal vector is d, and the communication round is divided into d time slots. In the jth time slot, the user equipment is trained to convert the jth dimension component of the local gradient signal vector Upload to the base station within the same time-frequency resources.

[0023] Furthermore, the specific process of the base station aggregating the local gradients of each training user device and updating the global model through the over-the-air calculation method is as follows:

[0024] The base station receives the following superimposed signals uploaded by each training user equipment:

[0025]

[0026] in, is the signal received by the base station in the j-th equally divided time slot of the t-th communication round, To transmit the j-th dimension component of the local gradient signal vector The power allocation factor of is the additive white Gaussian noise vector, σ 2 is the noise power; is the channel gain vector between the kth training user equipment and the base station;

[0027] Subsequently, the base station aggregates the local gradient using an over-the-air calculation method to obtain an estimated value of the j-th dimension component of the aggregated gradient signal vector: After obtaining all d dimensional components, the base station obtains the aggregated gradient signal vector From the aggregated gradient signal vector In the example, the base station recovers the aggregated gradient vector of federated meta-learning Then, the base station updates the global model according to the following calculation formula:

[0028] θ t+1 =θ t -βg t

[0029] Where β is the meta-learning rate.

[0030] Furthermore, the base station uses an aggregated beamforming vector b t Perform local gradient aggregation based on air calculation, where the aggregated beamforming vector b t Processed signal vector for:

[0031]

[0032] The base station obtains the signal of all d dimensional components After that, we get the aggregated gradient signal vector And recover the federated meta-learning aggregate gradient vector from it as follows:

[0033]

[0034] In a second aspect, the present invention further provides an optimization method applicable to the above-mentioned federated meta-learning system based on over-the-air computing, comprising the following steps:

[0035] When entering each preset communication round, the base station obtains the channel gain vector from each user equipment to the base station in the current communication round, and each user equipment obtains the local data sample to be sent and calculates the local gradient;

[0036] The base station constructs a first optimization problem of a federated meta-learning system based on over-the-air computing based on the channel gain vectors from each user device to itself, a preset convergence criterion, a preset power constraint for each user device, and an aggregated gradient error constraint. The base station solves the first optimization problem to obtain an optimization method for the federated meta-learning system based on over-the-air computing.

[0037] The base station controls each user device and itself according to the federated meta-learning system optimization method based on over-the-air computing, so that each user configures the transmission power according to the power allocation factor configuration scheme, and the base station configures the beamforming vector according to the beamforming vector configuration scheme. Each user device performs local training according to the federated meta-learning method, and then sends local gradients and local data samples to the base station to complete the federated meta-learning based on over-the-air computing.

[0038] Furthermore, the convergence criterion is the loss function F(θ t+1 ) and the loss function F(θ t ), as follows:

[0039] F(θ t+1 )-F(θ t )≤ψ t ({p k},b)

[0040] in,

[0041]

[0042]

[0043]

[0044] m is the number of base station receiving antennas, δ and B are non-negative constants, and L is a positive constant;

[0045] The power constraint of each user equipment is:

[0046] |p k | 2 ≤P max

[0047] Among them, P max To train the maximum transmit power of the user equipment;

[0048] The aggregate gradient error constraint is:

[0049] MSE≤ò

[0050] Where MSE is the mean square error between the actual aggregate gradient signal and the ideal aggregate gradient signal, and ò is the maximum tolerable mean square error value;

[0051] The first optimization problem in building a federated meta-learning system based on air computing is:

[0052]

[0053] The optimization method of the federated meta-learning system based on air computing includes: the power allocation factor {p k} configuration scheme and the configuration scheme of the base station beamforming vector b.

[0054] Furthermore, in order to solve the first optimization problem of the federated meta-learning system based on air computing, it is divided into two sub-optimization problems, wherein the two sub-optimization problems are: solving the base station beamforming vector b t The first sub-optimization problem of the configuration scheme, and solving the power allocation factor {p k The second sub-optimization problem of the configuration scheme of}.

[0055] Furthermore, the first sub-optimization problem for solving the configuration scheme of the base station beamforming vector b is:

[0056]

[0057] The constraints of this subproblem are:

[0058]

[0059] in,

[0060] The optimization variable of the first sub-optimization problem is the beamforming vector b of the base station aggregating the local gradient. The first sub-optimization problem is solved to obtain the beamforming vector b configuration scheme of the base station when aggregating the local gradient of federated learning.

[0061] Furthermore, the power allocation factor {p k The second sub-optimization problem of the configuration scheme is:

[0062]

[0063] The constraints of this subproblem are:

[0064] |p k | 2 ≤P max

[0065] MSE≤ò

[0066] The optimization variable of the second sub-optimization problem is the power allocation factor {p k}, solve the second sub-optimization problem and determine the power allocation scheme for each training user device when sending the local gradient.

[0067] In a third aspect, the present invention provides a control device applicable to the above-mentioned federated meta-learning system based on over-the-air computing, the device comprising:

[0068] A local learning module is used to complete meta-learning training locally on each training user device and calculate the local gradient of each training user device;

[0069] An information acquisition module is used to obtain the channel gain vector from each training user equipment to the base station in the current communication round;

[0070] The federated meta-learning system optimization module based on over-the-air computing is used to determine, in the current communication round, a power allocation factor configuration scheme for each training user device when sending the local gradient, and a beamforming vector configuration scheme for the base station when receiving the local gradient. The federated meta-learning system optimization module based on over-the-air computing includes two submodules:

[0071] a configuration scheme determination submodule for determining a configuration scheme for the power allocation factor of each training user device in the federated meta-learning system based on over-the-air computation in the current communication round, as well as a configuration scheme for the base station beamforming vector, based on the channel gain vector from each training user device to the base station, the preset power constraint conditions for each training user device, and the preset aggregated gradient error constraint conditions;

[0072] A configuration scheme execution submodule is used to control each training user equipment and base station to configure the corresponding power or beamforming vector according to the configuration scheme of the power allocation factor and the configuration scheme of the base station beamforming vector;

[0073] A local gradient aggregation module is used to control the base station to aggregate the local gradients in the superimposed signals received by the base station based on an over-the-air calculation method after completing the beamforming vector configuration to obtain an aggregated gradient;

[0074] The global model updating module is used to control the base station to update the global model based on the global gradient obtained in the current training round.

[0075] In a fourth aspect, the present invention further provides an electronic device, characterized in that it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0076] Memory for storing computer programs;

[0077] The processor is configured to implement any of the steps of the above-mentioned federated meta-learning method, system optimization method, and system control method based on over-the-air computing when executing a program stored in the memory.

[0078] In a fifth aspect, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned federated meta-learning methods, system optimization methods, and system control methods based on air computing.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] The present invention proposes a federated meta-learning system based on over-the-air computing. When training user devices in the system perform local training using a federated meta-learning method, a preset time interval for each training user device to complete local training, i.e., the total time of each local training cycle, can be pre-set. Consequently, after each local training cycle, each user device obtains its own local gradient. When each training user device transmits its local gradient, a preset time interval for each device to communicate with the base station, i.e., the total time of each communication round, can be pre-set. Consequently, upon entering the current communication round, the base station obtains the channel gain vector from each training user device to the base station in the current communication round and determines the local gradient to be transmitted. Furthermore, before each device transmits its local gradient, the base station obtains an optimization method for the federated meta-learning system based on over-the-air computing, based on the channel gain vector from each training user device to the base station, preset power constraints for each training user device, and aggregated gradient error constraints. The method includes: a configuration scheme for the power allocation factor of each training user device and a configuration scheme for each beamforming vector of the base station. The base station then controls each training user device and itself to configure the corresponding power or beamforming vector according to the optimization method for the federated meta-learning system based on over-the-air computing. In this way, each training user device can simultaneously transmit local gradients to the base station on the same time-frequency resources based on the power allocation factor configuration scheme. Furthermore, when the base station receives the superimposed signals of the local gradients from each training user device, it aggregates the local gradients of each training user device based on the beamforming vector configuration scheme and uses the aggregated gradients to update the global model. This completes federated meta-learning based on over-the-air computing within that communication round.

[0081] The transceiver structures of each training user device and base station are configured according to the federated meta-learning system architecture based on over-the-air computing. This allows each training user device to perform meta-learning training locally and calculate local gradients, which helps improve the generalization capabilities of the global model and overcome the impact of local data heterogeneity in the training user devices. Furthermore, the local gradients of each training user device are transmitted to the base station on the same time-frequency resources, helping to improve the system's spectrum resource utilization.

[0082] In addition, the preset power constraints of each training user device help to save user energy consumption under the premise of completing the federated meta-learning task based on air computing. Furthermore, due to the joint optimization of the transmission power configuration of each training user device and the beamforming vector of the base station, the federated meta-learning aggregation process based on air computing is globally optimized, which improves the convergence of federated meta-learning and achieves better federated meta-learning training effects. Based on this, the solution provided by the embodiment of the present invention can improve the training effect of federated learning under heterogeneous user local data, while making full use of the system's spectrum resources to achieve efficient aggregation of local gradients of each training user device. Compared with traditional federated learning and traditional federated meta-learning, the training effect is improved and the communication efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0084] Figure 1 A schematic diagram of the structure of a federated meta-learning system based on over-the-air computing provided in an embodiment of the present invention.

[0085] Figure 2 A schematic diagram of the flow of a federated meta-learning system based on over-the-air computing provided in an embodiment of the present invention.

[0086] Figure 3 A schematic diagram of the structure of a transmitter and receiver of a federated meta-learning system based on over-the-air computing provided by an embodiment of the present invention;

[0087] Figure 4 A flowchart of an optimization method for a federated meta-learning system based on over-the-air computing provided by an embodiment of the present invention;

[0088] Figure 5 A schematic diagram of a solution process for an optimization method of a federated meta-learning system based on over-the-air computing provided by an embodiment of the present invention;

[0089] Figure 6 A schematic diagram of the structure of a federated meta-learning system control device based on over-the-air computing provided by an embodiment of the present invention;

[0090] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to better understand the present technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

[0092] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the examples described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of the present invention.

[0093] In the related art, in traditional federated learning systems, the possible heterogeneity of user local data is not taken into account, thereby limiting the further improvement of the performance of the federated learning model. At the same time, the traditional federated meta-learning system ignores the impact of the wireless channel between the training user device and the base station, which in turn limits the application of the federated meta-learning system in wireless communication networks. Based on this, there is an urgent need for an improved federated meta-learning system, that is, a method that can improve the generalization performance of the global model in federated learning, overcome the adverse effects of user local data heterogeneity, and improve the efficiency of wireless communication between the training user device and the base station, so that when performing federated learning on the training user device, the adverse effects of data heterogeneity can be overcome, thereby improving the training effect of the global model and achieving efficient communication between the user and the base station.

[0094] In order to solve the above technical problems, the embodiment of the present invention provides a federated meta-learning system based on over-the-air computing. Figure 1 , including: K user equipment and a base station. Among them, K user equipment can be divided into K tra training user devices and K tst Each test user device is responsible for training the model using meta-learning methods and testing the model's performance and generalization capabilities. At the beginning of each training round, all users obtain the global model for the current training round from the base station. After obtaining the current global model, each training user completes local federated meta-learning training and uploads the local gradient to the base station. After obtaining the current global model, each test user tests the performance and generalization capabilities of the global model based on its own local dataset and no longer communicates with the base station during the current training round.

[0095] In the current t-th training round, K user devices have local data sets D1, D2, ..., D K The local data sets of each user device are heterogeneous, and the local data distribution between user devices is not independent and identically distributed.

[0096] Optionally, in a specific implementation, all the above-mentioned users obtain the global model of the current training round from the base station, which can be achieved by the base station sending the global model to all users by broadcasting at the beginning of each training round.

[0097] Optionally, in a specific implementation, all K user devices are divided into K tra training user devices and K tst The test user equipment can be completed by random division, wherein the number of training user equipment and training user equipment can be any number set according to the needs of actual application, and this embodiment of the present invention does not make specific limitations on this.

[0098] Optionally, in a specific implementation, the non-IID local user data may be expressed in the following form:

[0099] The sample size and category distribution of the local datasets held by training and testing user devices differ. The sample category distribution of the local datasets of the training user devices also differs. The local datasets of training user devices contain only a limited number of categories and are smaller in size, while the local datasets of testing user devices are larger and contain a complete range of categories.

[0100] The one training round includes a local training cycle and a communication round.

[0101] Among them, the kth training user device performs federated meta-learning training based on the local dataset and can obtain the local gradient Each training UE then uploads its local gradient to the base station within the same time-frequency resources. After receiving the superimposed signals from each training UE, the base station aggregates the local gradient information from each training UE to obtain the aggregated gradient for the current training round and uses it to update the global model.

[0102] The training of the federated meta-learning includes two gradient calculations. First, the k-th training user device combines all its local datasets D k Divide into training sets and validation set Then based on the training set Calculate the first gradient, perform a gradient descent, and obtain the localized update parameter. Then, the kth training user device updates the localized update parameter based on the validation set. Calculate the second gradient, the local gradient

[0103] Understandably, since traditional federated learning systems do not consider training on heterogeneous local data of user devices, in order to meet the demand for improving training performance on heterogeneous local data of user devices, it is necessary to rationally design the above-mentioned method for local training of user devices to support federated meta-learning based on over-the-air computing for federated learning tasks under heterogeneous local data. Based on this, the federated meta-learning method based on over-the-air computing provided by the embodiments of the present invention can be used to complete local gradient calculation of user devices based on heterogeneous local data, and aggregate local gradients using over-the-air computing methods at the base station.

[0104] Figure 2 The following is a flow chart of a federated meta-learning method for air computing provided by an embodiment of the present invention. Figure 2 As shown, the federated meta-learning method of over-the-air computing may include the following steps:

[0105] S201: When entering each preset local training cycle, each training user device and each test user device obtains global model parameters of the current local training cycle; each training user device divides its local data set into a training set and a test set;

[0106] A training round of the federated meta-learning system based on over-the-air computing includes a local training cycle and a communication round; wherein, each training user device and test user device completes a local training of the global model and a test of the global model performance in a local training cycle, and each training user device communicates with the base station once in a communication round.

[0107] The length of the local training cycle may be any length set according to the requirements of actual applications, and the embodiment of the present invention does not impose any specific limitation on this.

[0108] Each training user equipment and test user equipment can obtain the global model parameters of the current local training cycle in a variety of ways, which are not specifically limited in the embodiment of the present invention.

[0109] For example, the base station may broadcast the global model parameters of the current local training cycle to each training user equipment and test user equipment at the beginning of each local training cycle.

[0110] Each training user device needs to divide its local data set into a training set and a test set to perform the next step of gradient calculation. Each training user device can divide its local data set according to any division criteria. The embodiment of the present invention does not impose any specific restrictions on the division criteria.

[0111] Optionally, in a specific implementation, the training user device may divide its local data set into a training set and a test set. Specifically, the kth training user device divides its local data set D k Divide into training set and validation set The two are equal in size.

[0112] S202: Each training user device calculates a first gradient based on its own training set, and then performs a gradient descent on the global model parameters to obtain localized update parameters;

[0113] After obtaining the global model parameters for the current local training cycle and dividing the model into training and test sets, each training user can calculate the first gradient based on their respective training sets. Each training user can calculate the first gradient based on the training set using any calculation method, and the present embodiment does not impose any specific restrictions on the selection criteria.

[0114] Optionally, in a specific implementation, the first gradient can be expressed as the gradient calculation of the single sample loss function for each sample in the training set. Specifically, in the current t-th local training cycle, the k-th training user device obtains the global model parameter θ t , and based on the training set The first gradient is calculated by the following formula

[0115]

[0116] in, Is the training set The size of Is the training set The jth training sample; is the gradient of the loss function obtained on the jth data sample of the kth user equipment in the current training round. The loss function can be any function set according to the needs of the actual application, and this embodiment of the present invention does not specifically limit this.

[0117] After calculating the first gradient, each training user performs a gradient descent on the global model parameters to obtain the localized update parameters in order to calculate the second gradient.

[0118] Optionally, in a specific implementation, in the current t-th local training cycle, the k-th training user equipment calculates the localization update parameter according to the following formula:

[0119]

[0120] Where α is the local learning rate.

[0121] S203: Each training user device calculates a second gradient based on its own verification set, namely, a local gradient;.

[0122] After each training user calculates the localized update parameters, they can calculate the second gradient, which is the local gradient of the current federated meta-learning.

[0123] Optionally, in a specific implementation, in the current t-th local training cycle, the k-th training user can calculate the second gradient as follows

[0124]

[0125] in, For the kth user device in its validation set Upper pair parameters The gradient obtained by derivation is calculated in the same way as the gradient In order to save local computing resources on the user's device, For the parameters The first-order gradient of .

[0126] S204: Each training user equipment uploads a local gradient to the base station on the same time-frequency resource, and the base station aggregates the local gradients of each training user equipment using an over-the-air calculation method.

[0127] After obtaining the local gradient, each training user device uploads it to the base station on the same time-frequency resource. The base station aggregates the local gradients of each training user device using an over-the-air calculation method.

[0128] It is understandable that, since traditional federated meta-learning does not consider wireless communication between user devices and base stations, in order to implement the above-mentioned federated meta-learning method based on over-the-air computing, each training user device uploads its local gradient to the base station on the same time-frequency resources, and the base station aggregates the local gradients of each training user device using over-the-air computing methods, it is necessary to reasonably design the transmitter-receiver structures in the above-mentioned training user devices and base stations to support the transmission tasks of federated meta-learning based on over-the-air computing. Based on this, a transmitter-receiver structure provided by an example of the present invention can be used to complete the transmission and reception of local gradients between the training user device and the base station.

[0129] Figure 3 A schematic diagram of a transmitter-receiver structure suitable for federated meta-learning based on over-the-air computing provided in an embodiment of the present invention. Figure 3As shown, in the current t-th training round, the k-th training user device first trains based on the federated meta-learning method to obtain the local gradient Then, the kth training user equipment is assigned a power factor Allocate transmit power for the local gradient signal vector. The base station divides each communication round into d time slots. Within each time slot, the training user equipment sends one dimension of the local gradient signal vector, after transmit power control, to the base station on the same time-frequency resource.

[0130] In the jth time slot of the current tth communication round, the base station receives the superimposed signals from each training user equipment Specifically:

[0131]

[0132] in, To transmit the j-th dimension component of the local gradient signal vector The power allocation factor of is the additive white Gaussian noise vector, and the noise power is σ 2 ; is the channel gain vector between the kth user equipment and the base station; K tra is the total number of training users, K tra is a set of labels of all training users. Subsequently, the base station aggregates the local gradient using an over-the-air calculation method to obtain the estimated value of the j-th dimension component of the aggregated gradient signal vector After obtaining all d dimensional components, the base station obtains the aggregated gradient signal vector From the aggregated gradient signal vector In the example, the base station can recover the aggregated gradient vector of federated meta-learning It is assumed that each channel gain vector remains unchanged within a communication round.

[0133] Based on the superimposed signals received from various users The base station processes the superimposed signal using a beamforming vector to perform local gradient aggregation based on air calculation to obtain the federated meta-learning aggregate gradient

[0134] Specifically, the base station uses an aggregated beamforming vector b t For superimposed signals After processing, the aggregated beamforming vector b t Processed signal for:

[0135]

[0136] in, is the local gradient signal vector The base station accumulates and reconstructs the federated meta-learning aggregate gradient over d time slots. as follows:

[0137]

[0138] Then, the base station updates the global model according to the following formula:

[0139] θ t+1 =θ t -βg t

[0140] Among them, θ t is the vector composed of the current global model parameters, and β is the meta-learning rate.

[0141] Based on a transmitter-receiver structure of a federated meta-learning system based on over-the-air computing provided by an embodiment of the present invention, each training user device can use the same time-frequency resources to send a local gradient to a base station; after receiving the superimposed signal, the base station can aggregate the local gradients of each training user device based on an over-the-air computing method, thereby helping to improve the system's spectrum resource utilization and achieve efficient communication between the training user devices and the base station.

[0142] The aforementioned transceiver structure can be an improvement to the transceivers within existing user equipment and base stations. Furthermore, the transceiver structure provided by the embodiments of the present invention can also be applied to any scenario requiring user equipment to utilize the same time-frequency resources for transmission, to perform tasks other than federated meta-learning. This is not specifically limited in the embodiments of the present invention.

[0143] Corresponding to the above-mentioned federated meta-learning method based on over-the-air computing and a transmitter-receiver structure suitable for federated meta-learning based on over-the-air computing, an embodiment of the present invention provides a method for optimizing a federated meta-learning system based on over-the-air computing.

[0144] Figure 4 The flowchart of a method for optimizing a federated meta-learning system based on air computing provided by an embodiment of the present invention is as follows. Figure 4 As shown in Figure 2, the optimization method for the federated meta-learning system based on over-the-air computing can include the following steps:

[0145] S401: When entering each preset communication round, the base station obtains a channel gain vector from each training user equipment to the base station in the current communication round, and each training user equipment performs training according to the federated meta-learning method and calculates a local gradient;

[0146] A training round of the federated meta-learning system based on over-the-air computing includes a local training cycle and a communication round; wherein, each training user device uses the federated meta-learning method to perform local training in a local training cycle, and each training user device communicates with the base station once in a communication round; when performing the above-mentioned communication, the time interval for each user device to communicate with the base station can be preset, that is, the total length of each communication round, so that when entering each preset communication round, the base station can obtain the channel gain vector from each user device to the base station.

[0147] The length of the communication round may be any duration set according to the requirements of actual applications, and the embodiment of the present invention does not impose any specific limitation on this.

[0148] The base station can obtain the channel gain vector in various ways, which are not specifically limited in this embodiment of the present invention.

[0149] For example, the base station may obtain the channel gain vector using a channel estimation method, which may be any method capable of obtaining global channel state information (CSI). The embodiment of the present invention does not impose any specific limitation on the channel estimation method.

[0150] S402: The base station obtains a target over-the-air computation-based federated meta-learning system optimization method based on channel gain vectors from each training user equipment to itself, a preset convergence criterion, a preset power constraint of each training user equipment, and an aggregated gradient error constraint;

[0151] Among them, the optimization method of the federated meta-learning system based on air computing includes: the power allocation factor {p k} configuration scheme, the configuration scheme of the base station beamforming vector b; the preset convergence criteria include: the loss function F(θ t+1 ) and the loss function F(θ t ); the preset power constraints of each training user device include: the limited transmit power of each training user device in the current communication round; the aggregate gradient error constraint includes: in the current communication round, the tolerance of the base station for the aggregate gradient error when receiving user information does not exceed the threshold;

[0152] After the base station obtains the above-mentioned channel gain vector in the current communication round, the base station can determine the configuration of the power allocation factor of each user device in the federated meta-learning system based on over-the-air computing in the current communication round, as well as the configuration of the beamforming vector of the base station based on the obtained channel gain vector, the preset convergence criterion, the preset power constraints of each user device, and the aggregated gradient error constraints, thereby obtaining the target over-the-air computing-based federated meta-learning system optimization method in the current communication round.

[0153] The so-called convergence criterion is the loss function F(θ t+1 ) and the loss function F(θ t ) is the expected value of the difference.

[0154] Optionally, in a specific implementation, the above convergence criterion can be expressed as follows:

[0155] F(θ t+1 )-F(θ t )≤ψ t ({p k},b)

[0156] in,

[0157]

[0158]

[0159]

[0160] m is the number of base station receiving antennas, δ and B are non-negative constants, and L is a positive constant.

[0161] Based on this, in this specific implementation, in the obtained target-based federated meta-learning system optimization method based on air computing, the loss function F(θ t+1 ) and the loss function F(θ t ) has an upper bound on the expectation. By optimizing the power allocation factor of each training user device and the base station beamforming vector, the difference in the loss function of two adjacent communication rounds of federated meta-learning based on over-the-air computing can be reduced. Therefore, the convergence speed of federated meta-learning based on over-the-air computing can be accelerated and the performance of federated learning can be improved.

[0162] The preset power constraints for each training user device include the limited transmit power of each training user device within the current communication round. This means that in the resulting federated meta-learning system optimization method based on over-the-air computing, the transmit power of each training user device is limited. The power allocation factor should be properly configured to ensure that the transmit power of each training user device does not exceed the maximum transmit power.

[0163] Optionally, in a specific implementation, the power constraints of each training user equipment may include:

[0164] |p k | 2 ≤P max

[0165] Among them, p k is the power allocation factor when the kth training user equipment transmits the local gradient signal vector in the current communication round, P max is the maximum transmit power of the training user equipment. k∈K tra , K tra is a set of labels of all training users. This constraint describes the limited transmission power of the training user equipment in the system.

[0166] Based on this, in this specific implementation, in the obtained target-based federated meta-learning system optimization method for air computing, each user device configures the power allocation factor so that its own transmission power is not greater than P max .

[0167] The so-called aggregate gradient error constraint includes the following: within the current communication round, the base station must tolerate no more than a threshold value of aggregate gradient error when receiving user information. This means that in the resulting federated meta-learning system optimization method based on over-the-air computing, the base station can tolerate a certain error in the aggregate gradient of the federated meta-learning system when receiving user information due to the influence of the wireless channel, but the tolerance must not exceed a threshold.

[0168] Optionally, in a specific implementation, the above-mentioned aggregate gradient error constraint may include:

[0169] MSE≤ò

[0170] Where MSE is the mean square error between the actual aggregate gradient signal and the ideal aggregate gradient signal, and ò is the maximum tolerable mean square error value.

[0171] Based on this, in this specific implementation, in the obtained target-based federated meta-learning system optimization method for over-the-air computing, when the base station aggregates the local gradients of each device, the aggregated gradient error does not exceed ò.

[0172] S403: The base station controls each user device and itself according to the federated meta-learning system optimization method based on over-the-air computing, so that each user configures the transmission power according to the power allocation factor configuration scheme, and the base station configures the beamforming vector according to the beamforming vector configuration scheme, so that each user device sends a local gradient to the base station, completing the federated meta-learning based on over-the-air computing.

[0173] After obtaining the above-mentioned target optimization method for the federated meta-learning system based on air computing, the base station can control each user device to set the power allocation factor to the power allocation factor included in the target optimization method for the federated meta-learning system based on air computing, and control the base station to set its own beamforming vector according to the beamforming vector included in the target optimization method for the federated meta-learning system based on air computing.

[0174] In this way, each user device can send local gradients to the base station according to the power allocation factor included in the target over-the-air federated meta-learning system optimization method. The base station can then perform over-the-air aggregation of the local gradients uploaded by each user device according to the beamforming vector included in the target over-the-air federated meta-learning system optimization method.

[0175] Optionally, in a specific implementation method, after determining the above-mentioned target federated meta-learning optimization method based on air calculation, the base station can inform each user device of the power allocation factor configuration scheme by broadcasting, so that each user device can send a local gradient to the base station according to the power allocation factor included in the target federated meta-learning optimization method based on air calculation.

[0176] Furthermore, the preset power constraints for each user device help conserve energy while completing federated meta-learning based on over-the-air computing. Furthermore, by jointly optimizing the transmit power configuration of each user device and the base station's beamforming vector, the convergence process of federated meta-learning based on over-the-air computing is globally optimized, improving the convergence of federated meta-learning based on over-the-air computing and achieving better training results.

[0177] Based on this, the solution provided by the present invention can overcome the adverse effects of heterogeneous user device data, improving learning effectiveness compared to traditional federated learning. Furthermore, the solution can also improve the utilization efficiency of communication resources, enabling efficient communication between user devices and base stations.

[0178] Optionally, in a specific implementation, in step S402, the base station obtains a target federated meta-learning system optimization method based on over-the-air computing based on a channel gain vector from each training user equipment to itself, a preset convergence criterion, a preset power constraint of each user equipment, and an aggregated gradient error constraint, and may include the following step A:

[0179] Step A: The base station uses the acquired channel gain vectors from each training user device to itself, the preset convergence criterion, the preset power constraints of each user device, and the aggregated gradient error constraint to solve the preset over-the-air computing-based federated meta-learning first optimization problem and obtain the target over-the-air computing-based federated meta-learning system optimization method for the current communication round;

[0180] The first optimization problem of federated meta-learning based on air computing is:

[0181]

[0182] Among them, {p k} is the power allocation factor when the kth training user equipment transmits the local gradient signal vector, b is the beamforming vector used by the base station to receive the local gradient;

[0183]

[0184] in,

[0185]

[0186]

[0187] m is the number of base station receiving antennas, α is the local learning rate, β is the meta-learning rate, δ and B are non-negative constants, L is a positive constant, σ 2 is the power of the preset additive white Gaussian noise, K tra is the total number of training user devices, K tra is a set of labels of each training user device, is the loss function F(θ t ) for the current global model parameter vector θ t The gradient of θ, d is t The dimension size, and The sizes of the training and test sets for the k-th training user device, respectively.

[0188] Optionally, in a specific implementation, in step A above, the base station uses the obtained channel gain vectors from each training user equipment to itself, a preset convergence criterion, and the preset power constraints and aggregated gradient error constraints of each user equipment to solve the preset first optimization problem of federated meta-learning based on over-the-air computing to obtain the target in the current communication round. The optimization method of the federated meta-learning system based on over-the-air computing may include the following steps A1-A2:

[0189] Step A1: Split the preset first optimization problem of federated meta-learning based on over-the-air computing into a first sub-optimization problem and a second sub-optimization problem;

[0190] The optimization variable of the first sub-optimization problem is the beamforming vector b of the base station aggregated local gradient, and the optimization variable of the second sub-optimization problem is the power allocation factor {p k}.

[0191] Optionally, in a specific implementation, the first sub-optimization problem is:

[0192]

[0193] Among them, the first sub-optimization problem has constraints, specifically:

[0194]

[0195] in,

[0196] Optionally, in a specific implementation, the second sub-optimization problem is:

[0197]

[0198] Among them, the second sub-optimization problem has constraints, specifically:

[0199] |p k | 2 ≤P max

[0200] MSE≤ò

[0201] Furthermore, in each of the above specific implementations, both the first sub-optimization problem and the second sub-optimization problem must satisfy the above preset power constraints and aggregated gradient error constraints of each user equipment.

[0202] Based on this, optionally, in a specific implementation, both the first sub-optimization problem and the second sub-optimization problem have constraints: k | 2 ≤P max and MSE≤ò.

[0203] Step A2: The base station uses the obtained channel gain vectors from each training user device to itself, the preset convergence criterion, the preset power constraints of each user device, and the aggregated gradient error constraints to solve the first sub-optimization problem and the second sub-optimization problem respectively, and obtains the target of the current communication round based on the federated meta-learning system optimization method of air computing.

[0204] Among them, since the optimization variable of the first sub-optimization problem is the beamforming vector b of the base station aggregated local gradient, therefore, by solving the first sub-optimization equation, the configuration of the beamforming vector b of the base station aggregated local gradient in the target federated meta-learning system optimization method based on air computing can be obtained; since the optimization variable of the second sub-optimization problem is the power allocation factor {p k}, therefore, by solving the second sub-optimization problem, we can obtain the power allocation factor {p k} configuration scheme.

[0205] Figure 5 The following is a schematic diagram of a solution process of a specific federated meta-learning system optimization method based on air computing provided by an embodiment of the present invention. Figure 5 As shown, in a specific implementation, based on the solution process of the federated meta-learning system optimization method based on over-the-air computing, the first and second sub-optimization problems can be solved separately using a convex optimization method, which specifically includes the following steps:

[0206] S501: Solve the first sub-optimization problem using the interior point method to obtain a beamforming vector of the base station aggregated local gradient;

[0207] For the above-mentioned first sub-optimization problem, the interior point method can be optionally used to solve the first sub-optimization problem, thereby determining the beamforming vector b of the base station aggregated local gradient in the target federated meta-learning system optimization method based on air computing.

[0208] S502: Solve the second sub-optimization problem using an interior point method to obtain a power allocation factor for each user equipment.

[0209] For the second sub-optimization problem, the interior point method can be optionally used to solve the first sub-optimization problem, thereby determining the power allocation factor {p k}.

[0210] S503: If convergence is not achieved, continue to execute S501;

[0211] In the above-mentioned solution process, if no corresponding converged solution is obtained during the steps S501 and S502, that is, during the process of solving the first sub-optimization problem and the second sub-optimization problem respectively, the process returns to S501 and continues with S501 and S502; if both the first sub-optimization problem and the second sub-optimization problem obtain converged solutions, or the number of times the first sub-optimization problem and the second sub-optimization problem are solved reaches the preset number, the process proceeds to S504.

[0212] S504: If convergence is achieved, the optimization process ends;

[0213] In the above solution process, if the first and second sub-optimization problems are both solved to the corresponding convergence, it is equivalent to obtaining the target. In the optimization method of the federated meta-learning system based on air computing, the configuration of the beamforming vector b of the base station aggregated local gradient and the power allocation factor {p k} configuration scheme, the optimization process ends.

[0214] Corresponding to the above-mentioned embodiments of the present invention, which provide a federated meta-learning based on over-the-air computing, a transmitter-receiver structure suitable for a federated meta-learning system based on over-the-air computing, and an optimization method for a federated meta-learning system based on over-the-air computing, an embodiment of the present invention provides a control device suitable for a federated meta-learning system based on over-the-air computing.

[0215] Figure 6 The following is a schematic diagram of a control device for a federated meta-learning system based on air computing provided by an embodiment of the present invention. Figure 6 As shown, the control device of the federated meta-learning system based on air computing includes:

[0216] Local gradient acquisition module 610: used to control each training user device to complete federated meta-learning training and obtain local gradients in the current communication round;

[0217] An information acquisition module 620 is configured to acquire a channel gain vector from each training user equipment to the base station in the current communication round;

[0218] The federated meta-learning system optimization module 630 based on over-the-air computing is configured to determine, in the current communication round, a power allocation factor configuration scheme for each training user device when transmitting the local gradient, and a beamforming vector configuration scheme for the base station when receiving the local gradient and the local data sample. The federated meta-learning system optimization module based on over-the-air computing includes the following submodules:

[0219] The configuration scheme determination submodule is used to determine the configuration scheme of the power allocation factor of each training user device in the federated meta-learning system based on over-the-air computing in the current communication round, as well as the configuration scheme of the base station beamforming vector, based on the channel gain vector from each training user device to the base station, the preset power constraints of each training user device, and the preset aggregated gradient error constraints.

[0220] The configuration scheme execution submodule is used to control each training user equipment and base station to configure the corresponding power or beamforming vector according to the configuration scheme of the power allocation factor and the configuration scheme of the base station beamforming vector.

[0221] The local gradient aggregation module 640 is used to control the base station to aggregate the local gradients in the superimposed signals received by the base station based on an over-the-air calculation method after completing the beamforming vector configuration to obtain an aggregated gradient.

[0222] The global model updating module 650 is configured to control the base station to update the global model using the obtained aggregated gradient.

[0223] As can be seen from the above, by applying the solution provided by the embodiments of the present invention, when a training user device in a federated meta-learning system based on over-the-air computing performs local training using a federated meta-learning method, the time interval for each training user device to complete local training, i.e., the total time of each local training cycle, can be preset. Thus, after each local training cycle, each user device obtains its own local gradient. When each training user device sends a local gradient, the time interval for each device to communicate with the base station, i.e., the total time of each communication round, can be preset. Thus, upon entering the current communication round, the base station obtains the channel gain vector from each training user device to the base station in the current communication round and determines the local gradient to be sent. Furthermore, before each device sends its local gradient, the base station obtains an optimization method for the federated meta-learning system based on over-the-air computing based on the channel gain vector from each training user device to the base station, the preset power constraints for each training user device, and the aggregated gradient error constraint. The method includes: a configuration scheme for the power allocation factor of each training user device and a configuration scheme for each beamforming vector of the base station. Furthermore, the base station controls each training user device and itself to configure the corresponding power or beamforming vector according to the optimization method for the federated meta-learning system based on over-the-air computing. In this way, each training user device can simultaneously transmit local gradients to the base station on the same time-frequency resources based on the power allocation factor configuration scheme. Furthermore, when the base station receives the superimposed signals of the local gradients from each training user device, it aggregates the local gradients of each training user device based on the beamforming vector configuration scheme and uses the aggregated gradients to update the global model. This completes federated meta-learning based on over-the-air computing within that communication round.

[0224] The transceiver structures of each training user device and base station are configured according to the federated meta-learning system architecture based on over-the-air computing. This allows each training user device to perform meta-learning training locally and calculate local gradients, which helps improve the generalization capabilities of the global model and overcome the impact of local data heterogeneity in the training user devices. Furthermore, the local gradients of each training user device are transmitted to the base station on the same time-frequency resources, helping to improve the system's spectrum resource utilization.

[0225] In addition, the preset power constraints of each training user device help to save user energy consumption under the premise of completing the federated meta-learning task based on air computing. Furthermore, due to the joint optimization of the transmission power configuration of each training user device and the beamforming vector of the base station, the federated meta-learning aggregation process based on air computing is globally optimized, which improves the convergence of federated meta-learning and achieves better federated meta-learning training effects. Based on this, the solution provided by the embodiment of the present invention can improve the training effect of federated learning under heterogeneous user local data, while making full use of the system's spectrum resources to achieve efficient aggregation of local gradients of each training user device. Compared with traditional federated learning and traditional federated meta-learning, the training effect is improved and the communication efficiency is improved.

[0226] Optionally, in a specific implementation, the power constraint condition of each user equipment includes:

[0227] |p k | 2 ≤P max

[0228] Among them, P max is the maximum transmit power of the user equipment. This constraint describes the limited transmit power of the user equipment in the system.

[0229] The aggregate gradient error constraint may include:

[0230] MSE≤ò

[0231] Where MSE is the mean square error between the actual aggregate gradient signal and the ideal aggregate gradient signal, and ò is the maximum tolerable mean square error value.

[0232] Optionally, in a specific implementation, the federated meta-learning system optimization module based on over-the-air computing includes:

[0233] The configuration scheme determination submodule is used to use the obtained channel gain vectors from each user device to the base station, the preset power constraints of each user device and the preset aggregated gradient error constraints to solve the preset first optimization problem of federated meta-learning based on air computing, and obtain the optimization method of the federated meta-learning system based on air computing for the target in the current communication round, that is, to obtain the configuration scheme of the power allocation factor of each user device in the federated meta-learning system based on air computing in the current communication round, as well as the configuration scheme of the base station beamforming vector.

[0234] The configuration scheme execution submodule is used to control each user equipment and base station to configure the corresponding power or beamforming vector according to the configuration scheme of the power allocation factor and the configuration scheme of the base station beamforming vector.

[0235] The first optimization problem of federated meta-learning based on air computing is:

[0236]

[0237] Among them, {p k} is the power allocation factor when the kth training user equipment transmits the local gradient signal vector, b is the beamforming vector used by the base station to receive the local gradient;

[0238]

[0239] in,

[0240]

[0241]

[0242] m is the number of base station receiving antennas, δ and B are non-negative constants, and L is a positive constant.

[0243] Optionally, in a specific implementation, the configuration scheme determination submodule includes:

[0244] The solution splitting unit is used to split the preset federated meta-learning first optimization problem based on air computing into a first sub-optimization problem and a second sub-optimization problem; wherein the optimization variable of the first sub-optimization problem is the beamforming vector b of the base station aggregated local gradient, and the optimization variable of the second sub-optimization problem is the power allocation factor {p k}.

[0245] A solution solving unit is used to use the obtained channel gain vectors from each user device to the base station, the preset power constraints of each user device and the preset aggregated gradient error constraints to obtain a federated meta-learning system optimization method based on over-the-air computing for the target in the current communication round.

[0246] Optionally, in a specific implementation, the first sub-optimization problem is:

[0247]

[0248] The constraints of this subproblem are:

[0249]

[0250] in,

[0251] Optionally, in a specific implementation, the second sub-optimization problem is:

[0252]

[0253] The constraints of this subproblem are:

[0254] |p k | 2 ≤P max

[0255] MSE≤ò

[0256] Optionally, in a specific implementation, the solution-solving unit is specifically configured to:

[0257] according to Figure 5 The specific solution process of the federated meta-learning system optimization method based on over-the-air computing is shown in the figure. First, the interior point method is used to solve the first sub-optimization problem to obtain the beamforming vector of the base station's aggregated local gradient;

[0258] Then, the second sub-optimization problem is solved using the interior point method to obtain the power allocation factor of each training user equipment;

[0259] Determining whether a preset stopping condition is satisfied; wherein the preset stopping condition includes: the results obtained by solving the first sub-optimization equation and the second sub-optimization equation converge, or the number of times the first sub-optimization equation and the second sub-optimization equation are solved reaches a preset number;

[0260] If yes, the result obtained by solving the first sub-optimization equation and the second sub-optimization equation for the last time is used as the beamforming vector b of the base station aggregated local gradient and the power allocation factor {p of each user equipment in the federated meta-learning system optimization method based on air calculation under the current communication round. k};

[0261] Otherwise, return to the step of using the interior point method to solve the first sub-optimization problem and obtain the beamforming vector of the base station aggregated local gradient.

[0262] Corresponding to the above-mentioned embodiments of the present invention, which provide a federated meta-learning method based on over-the-air computing, a federated meta-learning system optimization method based on over-the-air computing, and a control device suitable for a federated meta-learning system based on over-the-air computing, an embodiment of the present invention also provides an electronic device.

[0263] Figure 7 An electronic device provided in an embodiment of the present invention includes:

[0264] like Figure 7 As shown, the electronic device includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other via the communication bus 704.

[0265] Memory 703, for storing computer programs;

[0266] The processor 701 is configured to implement the steps of any communication resource allocation method provided by the above-mentioned embodiments of the present invention when executing the program stored in the memory 703 .

[0267] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0268] The communication interface is used for communication between the above electronic device and other devices.

[0269] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0270] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0271] In another embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of any communication resource allocation method provided in the above embodiments of the present invention are implemented.

[0272] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is executed on a computer, the computer executes the steps of any communication resource allocation method provided in the above embodiments of the present invention.

[0273] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital terminal equipment line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0274] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0275] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, since the apparatus embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are generally similar to the method embodiments, their descriptions are relatively simple. For related portions, reference can be made to the descriptions of the method embodiments.

[0276] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A federated meta-learning system based on over-the-air computing, characterized by: The transceiver architecture includes at least one training user device and a base station. Each user device uses a power allocation factor to control the transmit power of a local gradient. The base station configures an aggregated beamforming vector to aggregate the local gradients based on over-the-air calculations. The following steps are employed: During each training round, the training user device responsible for local training divides its data samples into a training set and a test set. Subsequently, the training user device calculates the first gradient based on the samples of the training set and performs a gradient descent to obtain the local updated parameters; Then, the training user equipment calculates a second gradient based on the samples of the test set based on the local updated parameters, and uploads the second gradient to the base station; the base station aggregates the local gradients of each user equipment according to the over-the-air calculation method, and finally uses the aggregated gradient to update the global model; the training round includes a local training cycle and a communication round, and the uploading process of the local gradients of each training user equipment is carried out on the same time-frequency resources.

2. The federated meta-learning system based on over-the-air computing according to claim 1, characterized in that: The specific process of local federated meta-learning training on user devices is as follows: First, the kth training user device sends all its datasets D k Divide into training set and validation set Then based on the training set The first gradient is calculated as follows: Among them, θ t is the vector of current global model parameters broadcast by the base station to each user at the current t-th training round; When the current t-th training round is reached, the k-th training user device has Upper θ t The gradient obtained by derivation has a total of K tra Training user devices are trained for federated meta-learning, and the labels of each training user device constitute the set K tra ; Is the training set The size of Is the training set The jth training sample; is the loss function l of the kth training user device in the current training round k On its jth data sample, θ t The gradient obtained by differentiation; Next, each training user device performs a gradient descent, which is performed using the following formula: in, is the localized update parameter for the kth training user device, and α is the local learning rate; Then, the kth training user device is trained based on the validation set The second gradient is calculated according to the following formula: in, is the local gradient to be uploaded to the base station, is the number of users in the kth training set in the current tth training round Upper pair parameters The gradient obtained by derivation is calculated in the same way as the gradient resemblance; For the parameters The first-order gradient of The training user device converts the local gradient Perform preprocessing to obtain the local gradient signal vector to be uploaded as follows: Among them, φ is the preprocessing function, and the base station follows the gradient The dimension d of the local gradient signal vector is d, and the communication round is divided into d time slots. In the jth time slot, the user equipment is trained to convert the jth dimension component of the local gradient signal vector Upload to the base station within the same time-frequency resources.

3. The federated meta-learning system based on over-the-air computing according to claim 2, characterized in that: The specific process of the base station aggregating the local gradients of each training user device and updating the global model through over-the-air computing is as follows: The base station receives the following superimposed signals uploaded by each training user equipment: in, is the signal received by the base station in the j-th equally divided time slot of the t-th communication round, To transmit the j-th dimension component of the local gradient signal vector The power allocation factor of is the additive white Gaussian noise vector, σ 2 is the noise power; is the channel gain vector between the kth training user equipment and the base station; Subsequently, the base station aggregates the local gradient using an over-the-air calculation method to obtain an estimated value of the j-th dimension component of the aggregated gradient signal vector: After obtaining all d dimensional components, the base station obtains the aggregated gradient signal vector From the aggregated gradient signal vector In the example, the base station recovers the aggregated gradient vector of federated meta-learning Then, the base station updates the global model according to the following calculation formula: Where β is the meta-learning rate.

4. The federated meta-learning system based on over-the-air computing according to claim 3, characterized in that: The base station uses an aggregated beamforming vector b t Perform local gradient aggregation based on air calculation, where the aggregated beamforming vector b t Processed signal vector for: The base station obtains the signal of all d dimensional components After that, we get the aggregated gradient signal vector And recover the federated meta-learning aggregate gradient vector from it as follows:

5. An optimization method for a federated meta-learning system based on over-the-air computing according to any one of claims 1 to 4, characterized in that: The following steps are involved: When entering each preset communication round, the base station obtains the channel gain vector from each user equipment to the base station in the current communication round, and each user equipment obtains the local data sample to be sent and calculates the local gradient; The base station constructs a first optimization problem of a federated meta-learning system based on over-the-air computing based on the channel gain vectors from each user device to itself, a preset convergence criterion, a preset power constraint for each user device, and an aggregated gradient error constraint. The base station solves the first optimization problem to obtain an optimization method for the federated meta-learning system based on over-the-air computing. The base station controls each user device and itself according to the federated meta-learning system optimization method based on over-the-air computing, so that each user configures the transmission power according to the power allocation factor configuration scheme, and the base station configures the beamforming vector according to the beamforming vector configuration scheme. Each user device performs local training according to the federated meta-learning method, and then sends local gradients and local data samples to the base station to complete the federated meta-learning based on over-the-air computing.

6. The optimization method of the federated meta-learning system based on over-the-air computing according to claim 5, characterized in that: The convergence criterion is the loss function F(θ t+1 ) and the loss function F(θ t ), as follows: F(θ t+1 )-F(θ t )≤ψ t ({p k },b) in, m is the number of base station receiving antennas, δ and B are non-negative constants, and L is a positive constant; The power constraint of each user equipment is: |p k | 2 ≤P max Among them, P max To train the maximum transmit power of the user equipment; The aggregate gradient error constraint is: MSE≤ò Where MSE is the mean square error between the actual aggregate gradient signal and the ideal aggregate gradient signal, and ò is the maximum tolerable mean square error value; The first optimization problem in building a federated meta-learning system based on air computing is: The optimization method of the federated meta-learning system based on air computing includes: the power allocation factor {p k } configuration scheme and the configuration scheme of the base station beamforming vector b.

7. The optimization method of the federated meta-learning system based on over-the-air computing according to claim 6, characterized in that: In order to solve the first optimization problem of the federated meta-learning system based on air computing, it is divided into two sub-optimization problems, wherein the two sub-optimization problems are: solving the first sub-optimization problem of the configuration scheme of the base station beamforming vector b, and solving the power allocation factor {p k The second sub-optimization problem of the configuration scheme of}.

8. The optimization method of the federated meta-learning system based on over-the-air computing according to claim 7, characterized in that: The first sub-optimization problem for solving the configuration scheme of the base station beamforming vector b is: The constraints of this subproblem are: in The optimization variable of the first sub-optimization problem is the beamforming vector b of the base station aggregating the local gradient. The first sub-optimization problem is solved to obtain the beamforming vector b configuration scheme of the base station when aggregating the local gradient of federated learning.

9. The optimization method of the federated meta-learning system based on over-the-air computing according to claim 7, characterized in that: Solve the power allocation factor {p k The second sub-optimization problem of the configuration scheme is: The constraints of this subproblem are: |p k | 2 ≤P max MSE≤ò The optimization variable of the second sub-optimization problem is the power allocation factor {p k }, solve the second sub-optimization problem and determine the power allocation scheme for each training user device when sending the local gradient.

10. A control device for a federated meta-learning system based on over-the-air computing according to any one of claims 1 to 4, characterized in that: The device comprises: A local learning module is used to complete meta-learning training locally on each training user device and calculate the local gradient of each training user device; An information acquisition module is used to obtain the channel gain vector from each training user equipment to the base station in the current communication round; The federated meta-learning system optimization module based on over-the-air computing is used to determine, in the current communication round, a power allocation factor configuration scheme for each training user device when sending the local gradient, and a beamforming vector configuration scheme for the base station when receiving the local gradient. The federated meta-learning system optimization module based on over-the-air computing includes two submodules: a configuration scheme determination submodule for determining a configuration scheme for the power allocation factor of each training user device in the federated meta-learning system based on over-the-air computation in the current communication round, as well as a configuration scheme for the base station beamforming vector, based on the channel gain vector from each training user device to the base station, the preset power constraint conditions for each training user device, and the preset aggregated gradient error constraint conditions; A configuration scheme execution submodule is used to control each training user equipment and base station to configure the corresponding power or beamforming vector according to the configuration scheme of the power allocation factor and the configuration scheme of the base station beamforming vector; A local gradient aggregation module is used to control the base station to aggregate the local gradients in the superimposed signals received by the base station based on an over-the-air calculation method after completing the beamforming vector configuration to obtain an aggregated gradient; The global model updating module is used to control the base station to update the global model based on the global gradient obtained in the current training round.

Images