A federated learning efficient training optimization method for 6G network

Abstract

The application discloses a federated learning efficient training optimization method for a 6G network. In the downlink communication stage, global model parameters are distributed to clients by a parameter server. In the local training stage, a customized local training strategy is used to improve the convergence speed by training a specified number of iterations on the local client and adding a constraint term in the objective function to reduce the total number of communication rounds. In the uplink communication stage, some clients participate in the selection of clients with high calculation efficiency and upload their training results simultaneously, thereby reducing the communication cost of each round. In the model aggregation stage, the parameter server uses a weighted aggregation strategy based on data quality to improve model accuracy by assigning appropriate weights to different local parameters. Finally, the optimized global model parameters are output to realize efficient model training and parameter updating, providing an efficient and reliable solution for distributed learning in 6G networks.

Description

Technical Field

[0001] The invention relates to the field of communication network technology, and specifically to an efficient training optimization method for federated learning for 6G networks. Background Art

[0002] Globally, as exploration of the sixth-generation mobile communication technology (6G) continues to deepen, artificial intelligence (AI) is gaining widespread application in both daily life and industrial scenarios. 6G networks, with their ultra-high speeds, extremely low latency, and widespread connectivity, are expected to fully leverage vast amounts of sensitive data and enable a fully connected world. This vision will not only bring unprecedented convenience but also promote the integration of 6G with the inherent intelligence of AI. Furthermore, the high efficiency and reliability of 6G networks provide strong support for distributed computing. In 6G networks, data is typically stored across numerous edge devices, resulting in dispersed and massive data volumes. Traditional centralized data processing methods not only expose user privacy but also pose security risks during data transmission. Therefore, achieving efficient data processing and utilization while protecting user privacy has become a crucial challenge in the development of 6G networks.

[0003] To address this challenge, companies like Google and Apple have proposed federated learning. In this technique, clients train local deep neural network models using their private data. Local model updates are then sent to a central server, while the private data remains on the client. After collecting all local updates, the central server aggregates a new global model and provides it to the clients for the next round of model training. This distributed training iteration continues until the global model converges to a satisfactory test accuracy or reaches a specified number of iterations. Federated learning effectively protects client privacy because no private information is shared between the client and the central server.

[0004] Although federated learning offers significant advantages in communication efficiency and privacy protection, traditional federated learning still faces major challenges, such as excessive communication overhead and slow convergence. These issues stem from the following three aspects:

[0005] First, in the model training of federated learning, the data generated or collected by each client device in the network are different, and the statistical characteristics of these data obey non-independent identically distributed (Non-IID, non independent identically distributed), which means that the local data distribution P between different clients i, j is different. i≠P j This non-IID reduces the training efficiency of typical stochastic gradient descent (SGD)-based algorithms and seriously affects the convergence rate of the global model.

[0006] Secondly, traditional federated learning systems lack a strict mechanism for screening participants, typically employing a random strategy to select clients for aggregation. However, due to the heterogeneity, uncontrollability, and distributed nature of clients, using a random strategy to aggregate global models with clients can lead to an influx of low-quality clients, thereby undermining the overall performance of the federated learning system. Low-quality clients, due to poor data quality or the non-independent and identically distributed nature of the data used for training, can also lead to poor model performance.

[0007] Finally, the existing aggregation process in federated learning also significantly impacts overall communication overhead. Traditional aggregation methods, such as simple averaging strategies, while relatively straightforward to implement, often fail to fully consider the quality and diversity of individual client data. This simple aggregation strategy can limit the performance of the global model and fail to effectively utilize the data provided by each client, especially when data is unevenly distributed or the number of clients is large. Furthermore, simple averaging can overlook important information contributed by certain clients during training, thereby affecting the model's generalization ability.

[0008] Therefore, it is necessary to study an efficient federated learning training optimization method for 6G networks to reduce communication overhead while ensuring convergence speed. To achieve a better trade-off between local computation and communication overhead, we allow different clients to have different numbers of local training rounds, so that their update directions are as close as possible to the global model. During the model training phase, we adopt a customized local training strategy to reduce the number of communication rounds. By adding constraints to the objective function, we can optimize the number of local training rounds and improve convergence speed. Since each client has different computational efficiency, we select clients with high computational efficiency to participate in the aggregation, summarized as the "partial client participation rule", to reduce the communication overhead per round. In addition, we construct a data quality assessment model based on the client data volume, gradient similarity, and data diversity. We aggregate the local models with different weights calculated by the data quality assessment model into a global model, assigning appropriate weights to the local models, accelerating model convergence and improving the accuracy of the global model. Summary of the Invention

[0009] This invention provides an efficient training optimization method for federated learning for 6G networks, aiming to address the limitations of traditional federated learning in terms of communication efficiency and model convergence speed due to the heterogeneity of data distribution among different clients. This invention reduces the number of communication rounds through customized local training strategies, introduces partial client participation rules to reduce the cost of each round of communication, and adopts a weighted aggregation strategy based on data quality to improve model accuracy. While maintaining model accuracy, it can significantly reduce the number of communication rounds and participating clients in each round of communication, thereby reducing overall communication costs and accelerating model convergence.

[0010] A method for efficient training optimization of federated learning for 6G networks includes the following steps:

[0011] Step S1: Construct a federated learning training framework for 6G networks, which is used in scenarios where client data heterogeneity leads to huge communication costs and slow model convergence.

[0012] Step S2: Introduce a customized local training strategy to increase the convergence speed by adding constraints to the objective function of the local client to reduce the total number of communication rounds;

[0013] Step S3: Set the rules for some clients to participate, select clients with higher computational efficiency to upload their training results simultaneously, thereby reducing the communication cost of each round;

[0014] Step S4: Design a weight calculation method. In a non-IID setting, by analyzing three key factors: client data volume, gradient similarity, and data diversity, a data quality assessment model is constructed and the aggregation weights are adjusted to improve model accuracy.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] The present invention proposes an efficient training optimization method for federated learning for 6G networks. Because the data generated or collected by client devices is heterogeneous, measures need to be taken during the training process to alleviate this heterogeneity. Because adopting a random strategy to accept local models for aggregation will lead to an influx of low-quality clients, it is necessary to select high-quality clients to participate in the aggregation. Because the traditional average aggregation method fails to fully consider the quality and diversity of client data, a more reasonable weight calculation method needs to be designed. The federated learning training optimization method of the present invention considers the influencing factors more comprehensively, and provides an efficient and reliable solution for distributed learning in the 6G network environment, especially in processing large amounts of sensitive data and realizing a fully interconnected world. It has significant advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a training scenario diagram of the federated learning model provided by the present invention;

[0018] Figure 2 This is a flow chart of an efficient training optimization method for federated learning in 6G networks according to the present invention;

[0019] Figure 3 It is a schematic diagram comparing the effects of different Dirichlet distribution parameters;

[0020] Figure 4 This is a schematic diagram comparing the impact of the parameter ε of different local training strategies on accuracy;

[0021] Figure 5 This is a schematic diagram comparing the performance of different algorithms under actual Non-IID settings;

[0022] Figure 6 This is a schematic diagram of the performance comparison of different algorithms under the pathological Non-IID setting;

[0023] Figure 7 This is a schematic diagram comparing the communication overhead of 100 rounds of training for different algorithms. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that these examples are only used to illustrate the present invention and are not intended to limit the scope of the present invention. After reading the present invention, those skilled in the art will all fall within the scope defined by the appended claims to the modification of the various equivalent forms of the present invention.

[0025] The present invention is Figure 1 The federated learning model training system shown in the figure is used as an example. Figure 1 In the presented federated learning system model architecture, a central server is connected to multiple clients, with clients 1 through K each maintaining their own datasets and performing local training. During training, the server selects a subset of clients with high computational efficiency based on a data quality assessment model to participate in weighted aggregation of models to construct a global model. This data quality assessment model comprehensively considers the data volume, gradient similarity, and data diversity of each client. It evaluates data quality by calculating the KL divergence and assigns appropriate aggregation weights to each client's local model accordingly. Finally, the server distributes the aggregated global model parameters to all clients for the next round of local training and model updates. This process is repeated until the global model converges.

[0026] Based on the scene graph, the present invention proposes an efficient training optimization method for federated learning in 6G networks. In order to achieve efficient and accurate federated learning model training in 6G networks, this method specifically targets the different indicators of different clients in the 6G network. By combining customized local training strategies, partial client participation rules, and weighted aggregation strategies based on data quality, it effectively reduces communication overhead and accelerates the convergence of the model. This method not only reduces the number of communication rounds, but also optimizes the aggregation process of the global model through an accurate data quality assessment model, improves the accuracy and robustness of the model when processing non-independent and identically distributed data, and provides an efficient and reliable solution for distributed learning in future wireless communication networks. The method comprises the following steps:

[0027] Step S1: Downlink communication phase: The federated learning process starts with the downlink communication phase, which converts the global model parameters ω t Downloaded from the parameter server to K clients in round t.

[0028] Step S2: Local training phase: Each client computes an updated model based on the collected data. Taking into account the heterogeneity of data across clients, each client calculates the gradient of the loss function based on its own dataset and model parameters, and solves for the optimal number of local iterations, achieving higher accuracy with fewer iterations.

[0029] Step S3: Uplink Communication Phase: First, the computational efficiency of each client is evaluated based on the model training time and dataset size of the previous round. The model parameters of the n(t) clients with the highest computational efficiency are selected and uploaded to the parameter server. This phase directly determines the communication overhead of each round. After multiple communication rounds, the number of participants has a significant impact on the total communication overhead of this phase. The downlink and uplink communication phases constitute a complete communication round, which ends with the uplink communication round.

[0030] Step S4: Aggregation Phase: Each device transmits the latest model parameters to the parameter server. Once the transmission message arrives at the parameter server, the aggregation phase begins. The server waits for all participants to transmit data to the parameter server, calculates three different metrics, and constructs a data quality assessment model. Using data quality as a weight, it aggregates the model parameters of different clients. Finally, the aggregated global model is distributed back to the local client for the next round of local training.

[0031] Specific steps:

[0032] Step S1: At the initial stage of federated learning, the training dataset is partitioned. Different Dirichlet distribution parameters represent different data heterogeneity. The parameter server then sends the initial global model parameters to all clients participating in round t of training. This step is the initial communication link in the federated learning process, and its purpose is to ensure that all participating clients can start local training with the same model parameters.

[0033] In step S2, the client uses its private dataset to perform local model training. During this stage, each client calculates gradients based on the characteristics of its dataset and adjusts the number of local iterations based on data heterogeneity to achieve higher model accuracy with fewer iterations. This step is a key component of federated learning, as it involves the client utilizing local data to perform personalized optimization of the global model.

[0034] Step S3: After completing local training, clients participate in uplink communication based on their computational efficiency evaluation results. Only the n(t) clients with the highest computational efficiency are selected to upload their model parameters to the parameter server. This step determines the overhead of each communication round, as only a subset of clients participate in data upload, reducing the overall communication burden.

[0035] In step S4, after receiving the model parameters uploaded by the selected client, the parameter server enters the aggregation phase. The server calculates the quality of each client's data and uses these weights to weight and aggregate all uploaded model parameters. The aggregated global model parameters are then updated and distributed to all clients in preparation for the next round of federated learning. This step is the aggregation phase of federated learning, and its purpose is to integrate the model updates from each client into a global model update.

[0036] 1. To address the weight divergence between the local model and the global model caused by the heterogeneity of data distribution between different clients, a suitable local training strategy is specified for each client, providing different numbers of training cycles. At the same time, a constraint term is introduced to effectively reduce the gradient diversity between local loss functions, accelerate model convergence, and save communication costs. The specific steps are as follows:

[0037] (1) Adaptive local iteration strategy: Different devices in a federated network usually have different resource constraints in terms of computing hardware, network connectivity, and computing power levels. Therefore, it is unrealistic to force each device to perform the same number of local iterations as in the federated averaging algorithm. We allow a variable number of iteration cycles to be performed locally based on the system resources available to the device, and then aggregate the partial training results sent from stragglers, implicitly accommodating the different number of iterations of different devices. , we assume It is through Sub-local training The local model parameters of the client k, in This customized local training strategy achieves higher accuracy with fewer local iterations.

[0038] (2) Loss Function Constrained Optimization: While tolerating different amounts of work performed across devices can help mitigate the negative impact of system heterogeneity, excessive local updates may still potentially cause the method to diverge due to the underlying heterogeneous data. We propose to add a proximal term to the local subproblem to effectively constrain the impact of varying local updates. Specifically, instead of simply minimizing the local loss function, device k uses its selected local solver to approximately minimize the following objective equation:

[0039]

[0040] Among them, g k (ω k ;ω t ) represents the objective equation constructed after adding the regularization term to the local loss function. The problem is transformed from minimizing the local loss function to minimizing the objective equation g k (ω k ;ω t ), F k (ω k ) represents the loss function of client k, ε is the penalty factor of the constraint term, ω k represents the local model parameters of client k trained in round t, ω t Represents the global model parameters after the aggregation of the t-th round of training.

[0041] 2. In a 6G network environment, due to the limitation of uplink communication bandwidth, only a few clients can upload training results to the parameter server at the same time, which reduces the training speed and may cause the global model to favor clients with large data volumes. To address this problem, we propose a client selection strategy based on computational efficiency. By evaluating the computing power and data volume of the client, we select clients with high computational efficiency to participate in model training and parameter upload, thereby reducing communication overhead and ensuring that the global model evenly reflects the data characteristics of all clients, thereby improving the generalization ability and accuracy of the model. The specific steps are as follows:

[0042] (1) Client computing efficiency evaluation phase: using the local training time ζ of the previous round k and local dataset size |D k |, calculate the local computation efficiency e of client k k for:

[0043] Assume local data volume |D k | is constant, the local computation efficiency of client k in the tth communication round is expressed as where t latest is the most recent communication round, t latest <T. Since local training is not performed on all clients in the initialization phase, the local data volume of client k |D k | to initialize the local computing efficiency e k for:

[0044] In general, the client's initial local computing efficiency is much higher than its actual local computing efficiency, that is, Therefore, at the beginning of an iteration, clients with larger amounts of local data are more dispersed. In other words, clients that failed to complete training are more likely to be selected, thus quickly achieving true local computing efficiency.

[0045] (2) Client selection based on computational efficiency: Based on the information uploaded by the local client, the probability value p k It is used to represent the probability of a client being selected. Clients with higher computational efficiency tend to have larger p k , the parameter server calculates the probability value p for each client k As shown below:

[0046] In each round, the subset of selected clients is different, which makes the global model consider as much client information as possible. Therefore, the global loss function is transformed into the following formula:

[0047]

[0048] 3. In federated learning scenarios, the non-independent and identically distributed nature of client data poses challenges to global model accuracy and convergence. If data heterogeneity is not handled properly, it will reduce the accuracy of the global model and slow down the convergence rate. Existing methods often ignore this and mistakenly give too much weight to low-quality models, thereby affecting the performance of the global model. Therefore, we propose a new weighted aggregation method that evaluates data quality based on indicators such as data volume, gradient similarity, and data diversity, and uses KL divergence to quantify the differences between data sets to reasonably assign aggregation weights to local models. This method strengthens the impact of high-quality data, improves the accuracy and convergence of the global model, and maintains training efficiency. The specific steps are as follows:

[0049] (1) Calculation stages of different indicators:

[0050] a) We will client kThe data volume is defined as the normalized value of the ratio of client data to overall data, as shown in the following formula:

[0051]

[0052] b) In the federated learning scenario, the contribution of client k's data to the global model training can also be described by gradient similarity. Gradient similarity measures the data quality of client k by comparing the gradient of the local model with the global model gradient. From an optimization perspective, the main goal of federated learning is to obtain the optimal global model parameters by minimizing one or more loss functions. The gradient used in the fitting process reflects the direction of optimization to a certain extent. Therefore, if the training gradient generated by client k's data is closer to the assumed optimal gradient direction, it is considered to be more similar to the optimal model and the data quality is higher. Therefore, in the non-independent and identically distributed setting, we consider using gradient similarity to alleviate the client drift phenomenon. The gradient similarity γ of client k is constructed by aggregating and normalizing the cosine similarity of multiple rounds. k :

[0053]

[0054] in, represents the local model gradient vector of the client in round t, u t Represents the tth global model gradient vector, and iter represents the total number of gradient iterations.

[0055] c) When the data provided in federated learning is sufficiently reliable, we find that the diversity of data distribution can reflect the quality of the data to a certain extent. Higher data quality corresponds to richer data diversity. In the federated learning scenario, each data sample contains multiple feature dimensions. The Gram matrix G can calculate the feature representation vector x of the data sample. i ∈X k The volume of the parallelepiped constructed in Euclidean space. Previous research shows that the larger the volume of the constructed parallelepiped, the more divergent the data feature distribution, corresponding to higher data quality. Therefore, we use the Gram determinant to measure the diversity of the client's local data and normalize the data diversity as shown in the following formula:

[0056]

[0057] (2) Indicator weight evaluation stage: A data quality evaluation model including data volume, gradient similarity and data diversity is constructed through KL divergence to evaluate the impact of the above indicators.

[0058] set up γ and σ are the weights of data volume, gradient similarity and data diversity respectively. ρ γ and ρ σ To measure the importance of these three indicators. If the proportion of models with large data distribution skewness is reduced during the model aggregation process, the adverse impact of low-quality local models on the overall model performance can be reduced, and the accuracy of the global model can be improved. At the same time, if the data volume of multiple clients is the same, the data volume weight The influence of is not significant, so we can give an evaluation model for data quality, assuming that the probability distribution W is the target distribution, indicating that when the three indicators A set of weights when γ and σ are indistinguishable:

[0059]

[0060] (3) Data quality assessment model construction phase: Data quality assessment model C for client k k It is defined as follows:

[0061]

[0062] Model C is evaluated by client data quality k , the server collects all locally trained models from the clients and aggregates them using the normalization of the data quality assessment model as weights to update the global model; the t-round global model is obtained by:

[0063]

[0064] Among them, C k represents the data quality assessment model of client k, C i Represents the data quality assessment model for different clients i, where i is the client index. represents the local model parameters of client k in the tth round of training.

[0065] 4. We designed an efficient training optimization method for federated learning in 6G networks. To achieve efficient and accurate federated learning model training in 6G networks, this method specifically targets the different metrics of different clients in 6G networks. By combining customized local training strategies, partial client participation rules, and a weighted aggregation strategy based on data quality, it effectively reduces communication overhead and accelerates model convergence. This method not only reduces communication rounds but also optimizes the global model aggregation process through a precise data quality assessment model, improving the model's accuracy and robustness when processing non-IID data. This provides an efficient and reliable solution for distributed learning in future wireless communication networks.

[0066] The general setup of federated learning is that different clients initially have the same neural network model. These distributed devices iteratively train the local model a fixed number of times using their private data. A complete round of communication consists of the following stages:

[0067] (1) Downlink communication phase: The federated learning process starts with the downlink communication phase, which initializes the global model parameters ω t Downloaded from the parameter server to the K clients in round t as the starting point for model training.

[0068] (2) Local training phase: Each client calculates an updated model based on the collected data. Considering the data heterogeneity of different clients, each client calculates the gradient of the loss function based on its own dataset and model parameters, and solves for the optimal number of local iterations, achieving higher accuracy with fewer iterations.

[0069] (3) Uplink communication phase: First, the computational efficiency of each client is evaluated based on the model training time and dataset size of the previous round. The model parameters of the n(t) clients with the highest computational efficiency are selected and uploaded to the parameter server. This phase directly determines the size of the communication overhead in each round. After multiple communication rounds, the number of participants has a crucial impact on the total communication overhead of this phase. The downlink and uplink communication phases constitute a complete communication round, which ends with the uplink communication round.

[0070] (4) Aggregation phase: Each device transmits the latest model parameters to the parameter server. After the transmission message arrives at the parameter server, the aggregation phase begins. The server needs to wait for all participants to send transmission data to the parameter server, calculate three different indicators and build a data quality assessment model. The data quality assessment model is used as a weight to aggregate the model parameters of different clients. Finally, the aggregated global model is distributed back to the local client for the next round of local training.

[0071] First, we demonstrate how statistical heterogeneity affects convergence. We use two different dataset samples (FashionMNIST and CIFAR10) to generate samples for training using different Dirichlet distribution concentration parameters β to compare the performance of the algorithm under different heterogeneity conditions. The smaller the value of the Dirichlet parameter β, the higher the statistical heterogeneity of the generated sample labels.

[0072] pass Figure 3Simulation results show that as the Dirichlet distribution concentration parameter β decreases, that is, as data heterogeneity increases, the accuracy of the proposed method significantly improves and the convergence speed accelerates. This phenomenon demonstrates that the proposed method has significant advantages in processing non-IID data, especially when dealing with highly heterogeneous data distributions. Therefore, the proposed method has better performance for heterogeneous data and can effectively address the weight divergence caused by data heterogeneity in personalized federated learning.

[0073] Next, we examine the impact of different local training parameters on performance, fixing the local training period to E = 20. For different datasets, the local training parameter ε is set to 0, 0.3, 0.8, and 1 to represent different degrees of penalty.

[0074] Figure 4 Experimental results show that larger values ​​of ε when the penalty coefficient weight increases lead to better performance in various situations. Therefore, based on these results, ε was set to 1 in subsequent experiments. The experimental results also indicate that the customized local training strategy introduced in this method is applicable to real-world settings with varying statistical heterogeneity. Therefore, the method of this invention reduces the total number of communication rounds T and lowers the overall communication cost through the customized local training strategy.

[0075] The proposed CEFED method is compared with several related FL algorithms and compared with FedAvg, FedProx, SCAFFOLD, and PerAVG algorithms to highlight the advantages of this method. We compare the performance of CEFED and the baseline method in a practical non-IID setting with 100 rounds of communication. In order to intuitively demonstrate the performance of the CEFED algorithm, Figure 5 Figure 2 compares the test accuracy of CEFED with that of baseline methods over 100 iterations. The results show that on the Fashion-MNIST and CIFAR-10 datasets, the CEFED algorithm not only significantly outperforms all baseline methods but also exhibits more stable performance fluctuations. This stability is attributed to the consistency regularization term, which effectively mitigates the weight divergence problem caused by non-IID data, further demonstrating the superiority and practical value of CEFED in complex data environments.

[0076] In contrast, although the actual non-IID scenario based on Dirichlet partitioning has a more diverse and richer category distribution, which helps the model capture more diversified feature expressions and thus improves the generalization ability of the model, the fixed number of categories may limit the model's ability to accurately distinguish different client data distributions. Figure 6As shown in the figure, the comparison of the test accuracy between the proposed method and the benchmark method during 100 iterations is vividly demonstrated, further highlighting the performance advantage of CEFED in the pathological non-IID setting.

[0077] like Figure 7 As shown, a comparison of the communication overhead of different algorithms after 100 rounds of training on two data sets was also conducted. The CEFED algorithm proposed in the present invention demonstrated significant advantages in reducing communication overhead. By introducing a customized local training strategy, the algorithm can significantly reduce the required communication rounds while maintaining model accuracy. In addition, by implementing partial client participation rules, the CEFED algorithm allows only some clients to upload their local models in each round of training. This strategy effectively reduces the number of clients participating in each round of communication, thereby reducing communication costs. More importantly, based on the weighted aggregation strategy of data quality, the algorithm assigns reasonable weights to different local parameters by evaluating the amount of client data, gradient similarity, and data diversity, which not only improves model accuracy, but also reduces the number of times communication is required to correct the model.

[0078] The present invention discloses a federated learning efficient training optimization method for 6G networks, which includes three key modules: a customized local training strategy, partial client participation rules, and a weighted aggregation strategy based on data quality. The working process of the method is as follows: first, in the downlink communication phase, the global model parameters are sent to the client by the parameter server; second, in the local training phase, the customized local training strategy improves the convergence speed by training the local client for a specified number of iterations and adding constraints to the objective function to reduce the total number of communication rounds; then, in the uplink communication phase, the partial client participation rules select clients with higher computational efficiency and upload their training results at the same time, thereby reducing the communication cost of each round; finally, in the model aggregation phase, the parameter server improves the model accuracy by assigning reasonable weights to different local parameters based on the weighted aggregation strategy of data quality, and finally outputs the optimized global model parameters to achieve efficient model training and parameter update, providing an efficient and reliable solution for distributed learning in 6G networks.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for efficient training optimization of federated learning for 6G networks, comprising the following steps: Step S1: Construct a federated learning training framework for 6G networks, which is used in scenarios where client data heterogeneity leads to high communication costs and slow model convergence. Step S2: Introduce a customized local training strategy to increase the convergence speed by adding a constraint term to the objective function of the local client to reduce the total number of communication rounds; implement the customized local training strategy by allowing different iteration cycles to be executed locally across clients; introduce a constraint term to effectively reduce the gradient diversity between local functions; client k approximately minimizes the following objective equation using the local loss function: Among them, g k (ω k ;ω t ) represents the target equation constructed after adding the regularization term to the local loss function, F k (ω k ) represents the loss function of client k, ε is the penalty factor of the constraint term, ω k represents the local model parameters of client k trained in round t, ω t Represents the global model parameters after the t-th round of training aggregation; Step S3: Set the rules for some clients to participate, select clients with higher computational efficiency and upload their training results at the same time, so as to reduce the communication cost of each round; the specific process of the rules for some clients to participate is as follows: in each round of communication, a subset of n(t) clients will be selected, denoted as S t The local model on the client is used to average the global model loss function after the aggregation stage on each selected client. Optimize the local objective function; where D k represents the amount of data of client k, and D represents the total amount of data. In the uplink communication phase, the client uploads the local model parameters and transmits the size of the local dataset and the training time of the local model in the previous round of communication to the parameter server. Step S4: Design a weight calculation method. In a non-IID setting, by analyzing three key factors: client data volume, gradient similarity, and data diversity, a data quality assessment model is constructed and the aggregation weights are adjusted to improve model accuracy.

2. The efficient training optimization method for federated learning in 6G networks according to claim 1 is characterized in that: In step S1 above, federated learning involves different clients initially using the same neural network model. Distributed devices composed of different clients iteratively train the local model a fixed number of times using their private data. A complete round of communication includes the following stages: Step S11: Downlink communication phase: The federated learning process starts with the downlink communication phase, which converts the global model parameters ω t Download from the parameter server to the K clients in round t; Step S12: Local training phase: Each client calculates an updated model based on the collected data. Considering the data heterogeneity of different clients, each client calculates the gradient of the loss function based on its own dataset and model parameters, and solves the optimal number of local iterations to achieve higher accuracy with fewer iterations. Step S13: Uplink communication phase: First, the computational efficiency of each client is evaluated based on the model training time and dataset size of the previous round, and the model parameters of the n(t) clients with the highest computational efficiency are selected and uploaded to the parameter server; The downlink and uplink communication phases constitute a complete communication round; Step S14: Aggregation phase: Each device transmits the latest model parameters to the parameter server; After the transmitted message arrives at the parameter server, the aggregation phase begins; The server needs to wait for the transmission data sent by all participants to the parameter server, calculate three different indicators and build a data quality assessment model, use data quality as a weight to aggregate the model parameters of different clients, and finally distribute the updated output back to the device.

3. The method for efficient training optimization of federated learning for 6G networks according to claim 2, characterized in that: In the above step S2, for the target equation, the variable θ represents different local iteration times. Indicates the iteration on client k in the t-th federated training times; assuming It is through The local model parameters of client k after local training, in 4. The method for efficient training optimization of federated learning for 6G networks according to claim 3, characterized in that: In the above step S3, the local training time ζ of the previous round is used k and local dataset size |D k |, calculate the local computation efficiency e of client k k for: Probability value p k Used to represent the probability of a client being selected. The parameter server calculates the probability value p for each client. k As shown below: In each round, the subset of selected clients is different, and the expression of the global loss function is transformed into the following:

5. The method for efficient training optimization of federated learning for 6G networks according to claim 4, characterized in that: In the above step S4, the quality of the data is first evaluated, and then the KL divergence-based model is used to determine the data relevance, and the aggregation weight of the model in federated learning is adjusted accordingly; the specific indicator is the normalized value of the data volume Gradient similarity normalization value γ k , the normalized value of data diversity σ k ; The data volume of client k is defined as the normalized value of the ratio of client data to total data Gradient similarity is used to alleviate client drift. The gradient similarity γ of client k is constructed by aggregating and normalizing the cosine similarity of multiple rounds. k : in, represents the local model gradient vector of the client in round t, u t represents the tth global model gradient vector, and iter represents the total number of gradient iterations; Use Gram's determinant to measure the diversity of client local data σ k , and normalize the data diversity as shown below:

6. The method for efficient training optimization of federated learning for 6G networks according to claim 5, characterized in that: In the above step S4, a data quality assessment model including data volume, gradient similarity and data diversity is constructed through KL divergence; γ and σ are the weights of data volume, gradient similarity and data diversity, respectively, and are expressed as ρ γ and ρ σ To measure the importance of these three indicators; Provide an evaluation model for data quality Assume that the probability distribution W is the target distribution, which means that when the three indicators A set of weights when γ and σ are indistinguishable.

7. The method for efficient training optimization of federated learning for 6G networks according to claim 6, characterized in that: In the above step S4, the data quality model C of client k k As shown below: Model C is evaluated by client data quality k ,The server collects all locally trained models from clients and aggregates them with the normalization of the data quality assessment model as weights to update the global model; The t-round global model is obtained as follows: Among them, C k represents the data quality assessment model of client k, C i Represents the data quality assessment model for different clients i, where i is the client index. represents the local model parameters of client k in the tth round of training.

Images