Wireless federal learning method based on characteristic norm regularization and statistical aggregation
The wireless federated learning method based on feature norm regularization and statistical aggregation solves the problems of data heterogeneity and unreliable communication in wireless communication environments, improves model accuracy and convergence speed, and is suitable for resource allocation and personalized services in wireless communication networks.
Patent Information
- Application Number
- CN202510950497.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-10
- Publication Date
- 2025-10-28
AI Technical Summary
Existing federated learning is difficult to simultaneously solve the problems of decreased model accuracy and slow convergence caused by data heterogeneity and unreliable communication in wireless communication environments.
By adopting the method of feature norm regularization and statistical aggregation, the feature norm regularization term is introduced in the local training of the client, the difference between the sample feature norm and the label average feature norm is dynamically calculated, and the global model update is optimized by combining model residual upload and statistical weighted aggregation.
It effectively alleviates model bias caused by data heterogeneity, enhances aggregation robustness under unreliable communication, improves model accuracy and convergence speed, and is suitable for resource allocation, personalized services and edge computing in wireless communication networks.
Smart Images
Figure CN120851139A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of federated learning technology and relates to a wireless federated learning method based on feature norm regularization and statistical aggregation. Background Technology
[0002] Federated learning provides an effective distributed learning solution for applications such as resource allocation and personalized services in wireless communication networks by collaboratively training machine learning models among distributed clients. Within this framework, clients train local models using local data and upload the model parameters to an aggregator for global model updates, thus avoiding the direct transmission of raw data and protecting data privacy.
[0003] However, federated learning faces significant challenges in real-world wireless communication environments. First, client-side local data is often heterogeneous, exhibiting variations in label distribution (label offset) or feature distribution under the same label (domain offset). This heterogeneity leads to inconsistent feature distributions in the client-side local model, impacting the accuracy and convergence speed of the global model. Second, wireless communication links are susceptible to time-varying channel conditions and interference, frequently resulting in the failure to transmit local model parameters. This unreliable communication causes incomplete model parameters received by the aggregator, leading to global model aggregation bias and further exacerbating model performance degradation.
[0004] In existing technologies, some methods address the issue of data heterogeneity by aggregating client model parameters through weighted averaging or by introducing regularization terms to constrain local model updates, thus mitigating the impact of label offset. However, these methods have limited effectiveness in handling domain offset and struggle to effectively align feature distribution differences among clients. On the other hand, regarding unreliable communication, some studies enhance the robustness of model aggregation by adjusting aggregation weights or considering client transmission reliability, but these methods typically do not adequately address the unreliable communication problem. Therefore, existing technologies struggle to simultaneously address the dual challenges of data heterogeneity and unreliable communication, limiting the performance and application of federated learning in complex wireless communication environments.
[0005] Therefore, there is an urgent need for a federated learning method that can simultaneously mitigate the impact of data heterogeneity and unreliable communication, in order to improve model accuracy, convergence speed and robustness of the aggregation process, and provide a more efficient solution for distributed learning applications in wireless communication networks. Summary of the Invention
[0006] In view of this, and considering the shortcomings of existing federated learning techniques in terms of decreased model accuracy and slow convergence speed under heterogeneous data and unreliable communication environments, the purpose of this invention is to provide a wireless federated learning method based on feature norm regularization and statistical aggregation. Feature norm regularization alleviates local model bias caused by data heterogeneity, and statistical weighted aggregation and model residual uploading enhance the robustness of the global model aggregation process under unreliable communication, effectively solving the global model performance problems caused by data heterogeneity and unreliable communication.
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] A wireless federated learning method based on feature norm regularization and statistical aggregation, comprising:
[0009] In wireless communication networks, base stations or cloud servers are regarded as aggregators, and each communication terminal is regarded as a client to establish a federated learning system. A global model and a local model with the same architecture are established in the aggregator and the client respectively.
[0010] The aggregator broadcasts the current global model parameters and feature norm differences to all clients, and the clients receive the parameters and initialize their local models.
[0011] The client calculates the sample feature norm and the label average feature norm through the feature extractor, constructs a regularization term, optimizes the loss function, performs local training, generates model residuals using the updated local model, and uploads them to the aggregator along with the label average feature norm.
[0012] The aggregator receives the residuals and feature norms uploaded by the clients, calculates the differences in feature norms between clients, performs statistical weighted aggregation based on the probability of successful transmission, and updates the global model.
[0013] The aggregator broadcasts the updated global model parameters and feature norm differences to the client via a wireless channel, and then proceeds to the next round of training. The entire process involves multiple rounds of iterative optimization of the global model until convergence.
[0014] Furthermore, the aggregator of the federated learning system is a base station or cloud server in the wireless communication network, and the client is various terminals in the wireless communication network, including smartphones, IoT sensors, and edge computing nodes.
[0015] The aggregator configures the global model w0 and initializes the label average feature norm difference. Configure system hyperparameters in the aggregator; hyperparameters include the learning rate η. i The parameters are: regularization parameter λ, total number of communication rounds M, number of clients I; client i configures the local model. Feature extractor and local dataset D iWherein, the superscript or subscript "i" represents client i; the subscript "t" represents the current communication round t, and the first round t = 0.
[0016] Furthermore, the aggregator broadcasts the global model parameter w to all clients. t and characteristic norm difference In the first round of training, the global model parameters w in the aggregator... t and characteristic norm difference w0 and In subsequent rounds, the aggregator uses the average feature norm from all client feedback. Update feature norm difference Based on the model residuals reported by all clients Update global model parameters w t After the global model is updated, w will be t+1 and Distribute to clients.
[0017] Furthermore, the aggregator calculates the average feature norm based on feedback from all clients. Update feature norm difference The process is as follows: The aggregator calculates the feature norm difference for each label k.
[0018]
[0019] In the formula, This represents the average feature norm of the feedback from client m, excluding client i.
[0020] The aggregator uses the model residuals from all clients as input. Update global model parameters w t The process is as follows: based on the transmission success probability p i For successfully uploaded model residuals Perform weighted aggregation and update the global model:
[0021]
[0022] In the formula, This represents the set of clients that successfully uploaded data in the current communication round t. For indicator functions, if Then α i =1, otherwise a i =0; β i w represents the weight of the model residual for each client i. t+1 These are the updated global model parameters; where the probability of successful transmission p i Obtained through historical channel statistics:
[0023]
[0024] In the formula, T represents the time scale.
[0025] Furthermore, the client receives the global model parameter w broadcast by the aggregator. t and characteristic norm difference For local model training, the client's local dataset originates from mobile devices, IoT devices, or edge computing nodes, and is data in the form of images, represented as follows: in, Indicates local sample labels. For local samples, N i Let j represent the number of local samples in client i, and j represent the j-th local sample.
[0026] The client-side feature extractor uses a convolutional neural network or a recurrent neural network, which is represented as follows: It will use local samples Extracting feature norms from it as input These are the parameters for the feature extractor.
[0027] The client first extracts the feature norm using a feature extractor. Then, the average feature norm is calculated for each label k. Next, the difference in the received feature norms is combined Construct regularization terms Thus, a local training loss function is established. Train the local model to obtain the updated local model. Calculate the model residuals based on the updated local model. Together Upload to the aggregator.
[0028] Furthermore, the feature norm calculated by client i Represented as:
[0029]
[0030] Based on the characteristic norm Calculate the average feature norm for each label k. The process is as follows:
[0031]
[0032] in, For indicator functions, when At that time, α j ￠=1, otherwise α j ￠=0.
[0033] Furthermore, client i determines the difference based on the received feature norm. Constructed regularization terms for:
[0034]
[0035] in, Let K represent the proportion of label k in the dataset of client i, where K is the total number of labels in the dataset;
[0036] The local training loss function Represented as:
[0037]
[0038] in, The cross-entropy loss is used, and λ is the feature norm regularization parameter.
[0039] Therefore, gradient descent is used to update the local model, and the process is expressed as follows:
[0040]
[0041] Where, η i This represents the learning rate of client i. This represents the gradient operator.
[0042] The beneficial effects of this invention are as follows:
[0043] This invention introduces a feature norm regularization term during local training on the client side. It dynamically calculates the difference between the sample feature norm and the average feature norm of the labels, embedding this difference into the loss function to constrain model training. This mechanism achieves feature space alignment without requiring original data sharing, effectively mitigating model bias caused by data distribution differences such as label skewness and missing feature dimensions.
[0044] To address unreliable factors such as packet loss and latency in wireless communication, this invention proposes a joint optimization strategy of residual uploading and statistical weighted aggregation. The client only uploads the model residual, combining it with the label's average feature norm as auxiliary information, thus reducing the communication load. The aggregator dynamically allocates weights based on the client's historical transmission success rate, giving higher priority to updates from highly reliable clients. Residual uploading and statistical aggregation enhance the aggregation robustness under unreliable wireless communication conditions.
[0045] Overall, this invention effectively aligns the feature distribution of client models through feature norm regularization, mitigating local model bias caused by data heterogeneity. It enhances the robustness of the aggregation process in unreliable communication environments through statistical weighted aggregation and model residual uploading, significantly improving the accuracy and convergence speed of federated learning in heterogeneous data and unreliable communication environments. It is applicable to resource allocation, personalized services, and edge computing in wireless communication networks, and has broad application prospects.
[0046] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description
[0047] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:
[0048] Figure 1 This is a schematic diagram of the overall process of the wireless federated learning method based on feature norm regularization and statistical aggregation in an embodiment of the present invention;
[0049] Figure 2 This is a schematic diagram comparing the wireless federated learning method based on feature norm regularization and statistical aggregation in an embodiment of the present invention with the traditional federated learning method.
[0050] Figure 3 This is a schematic diagram of the training process of the client-side local model according to an embodiment of the present invention;
[0051] Figure 4 This is a schematic diagram of the training process of the global aggregator model in an embodiment of the present invention. Detailed Implementation
[0052] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0053] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0054] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0055] Please see Figures 1-4 This is a wireless federated learning method based on feature norm regularization and statistical aggregation.
[0056] This embodiment details the specific process of a wireless federated learning method based on feature norm regularization and statistical aggregation, such as... Figure 1 As shown, it includes:
[0057] S1. In a wireless communication network, base stations or cloud servers are regarded as aggregators, and each communication terminal is regarded as a client to establish a federated learning system. A global model and a local model with the same architecture are established in the aggregator and the client respectively.
[0058] S2. The aggregator broadcasts the current global model parameters and feature norm differences to all clients. The clients receive the parameters and initialize their local models.
[0059] S3. The client calculates the sample feature norm and the label average feature norm through the feature extractor, constructs a regularization term, optimizes the loss function, performs local training, generates model residuals using the updated local model, and uploads them to the aggregator along with the label average feature norm.
[0060] S4: The aggregator receives the residuals and feature norms uploaded by the clients, calculates the differences in feature norms between clients, performs statistical weighted aggregation based on the probability of successful transmission, and updates the global model;
[0061] S5. The aggregator broadcasts the updated global model parameters and feature norm differences to the client via a wireless channel, and then proceeds to the next round of training. The entire process iterates through multiple rounds to optimize the global model until it converges.
[0062] like Figure 2 The diagram illustrates a comparison between a wireless federated learning method based on feature norm regularization and statistical aggregation, as proposed in this invention, and a traditional wireless federated learning method. Compared to traditional methods, this method effectively mitigates local model bias caused by label and domain offsets through feature norm regularization. For unreliable communication, statistical weighted aggregation and model residual uploading are employed to reduce bias caused by transmission failures and enhance the robustness of the aggregation process.
[0063] In step S1 of this embodiment, the aggregator of the federated learning system is a base station or cloud server in the wireless communication network, and the client is various terminals in the wireless communication network, including smartphones, IoT sensors, edge computing nodes, etc.
[0064] The aggregator configures the global model w0 and initializes the label average feature norm difference. Configure system hyperparameters in the aggregator; hyperparameters include the learning rate η. i Regularization parameter λ, communication rounds M, number of clients I.
[0065] Client i configures local model Feature extractor and local dataset Feature extractor Use convolutional neural networks (suitable for image classification) or recurrent neural networks. Local dataset D i The data originates from mobile devices, IoT devices, or edge computing nodes, and is data type: image data. The superscript or subscript "i" represents client i; the subscript "t" represents the current communication round t, with the first round t = 0. Indicates local sample labels. For local samples, N i Let represent the number of local samples in client i, and j represent the j-th local sample.
[0066] In step S2 of this embodiment, the aggregator broadcasts the global model parameter w to all clients. t and characteristic norm difference Client receives and stores To coordinate local training.
[0067] In step S3 of this embodiment, as Figure 3 As shown, the client's training process specifically includes:
[0068] Client i first utilizes the feature extractor For local samples Calculate the characteristic norm
[0069]
[0070] in, These are the parameters for the feature extractor.
[0071] Then, client i calculates the average feature norm for each label k.
[0072]
[0073] in, For indicator functions, when At that time, α j ￠=1, otherwise α j ￠=0.
[0074] Next, client i determines the difference based on the received feature norm. Construct regularization terms Among them, the regularization term Represented as:
[0075]
[0076] in, Let K represent the proportion of label k in the dataset of client i, where K is the total number of labels in the dataset.
[0077] Then, client i calculates the local training loss function.
[0078]
[0079] in, λ is the cross-entropy loss, and λ is the feature norm regularization parameter.
[0080] Next, gradient descent is used to update the local model:
[0081]
[0082] Where, η i This represents the learning rate of client i. This represents the gradient operator.
[0083] Finally, the model residuals are calculated based on the updated local model. Together Upload to the aggregator.
[0084] In step S4 of this embodiment, the aggregator calculates the feature norm difference for each label k.
[0085]
[0086] in, This represents the average feature norm of the feedback from client m, excluding client i.
[0087] And based on the transmission success probability p i The residuals from successfully uploaded samples are weighted and aggregated to update the global model.
[0088]
[0089] in, This represents the set of clients that successfully uploaded data in the current communication round t. This can be obtained through historical channel statistics; after the model is updated, w will be... t+1 and Distribute to clients.
[0090] In step S5 of this embodiment, the aggregator broadcasts the updated global model parameters and feature norm differences to the client via a wireless channel, entering the next round of training. The entire process iteratively optimizes the global model through multiple rounds until convergence. Specifically, as follows... Figure 4 As shown, the process includes:
[0091] Initialize the global model parameters and the difference in the average feature norm of the labels, and set the learning rate, number of communication rounds, and regularization hyperparameters;
[0092] The aggregator broadcasts the current global parameters;
[0093] The aggregator receives the model residuals and label average feature norms from successfully uploaded clients;
[0094] The aggregator calculates the client feature norm difference based on the average feature norm of the received tags;
[0095] The aggregator performs statistical weighted aggregation on the residuals of successfully uploaded data based on the client's transmission success probability, and updates the global model parameters.
[0096] Determine if the set number of communication rounds has been reached. If not, broadcast the current global parameters again; if so, training ends.
[0097] This invention effectively aligns the feature distribution of client models through feature norm regularization, mitigating local model bias caused by data heterogeneity. It enhances the robustness of the aggregation process in unreliable communication environments through statistical weighted aggregation and model residual uploading, significantly improving the accuracy and convergence speed of federated learning in heterogeneous data and unreliable communication environments. It is applicable to resource allocation, personalized services, and edge computing in wireless communication networks, and has broad application prospects.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A wireless federated learning method based on feature norm regularization and statistical aggregation, characterized in that: The method includes: In wireless communication networks, base stations or cloud servers are regarded as aggregators, and each communication terminal is regarded as a client to establish a federated learning system. A global model and a local model with the same architecture are established in the aggregator and the client respectively. The aggregator broadcasts the current global model parameters and feature norm differences to all clients, and the clients receive the parameters and initialize their local models. The client calculates the sample feature norm through the feature extractor to construct a regularization term, optimizes the loss function, performs local training, generates model residuals using the updated local model, and uploads them to the aggregator along with the label average feature norm. The aggregator receives the residuals and feature norms uploaded by the clients, calculates the differences in feature norms between clients, performs statistical weighted aggregation based on the probability of successful transmission, and updates the global model. The aggregator broadcasts the updated global model parameters and feature norm differences to the client via a wireless channel, and then proceeds to the next round of training. The entire process involves multiple rounds of iterative optimization of the global model until convergence.
2. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 1, characterized in that: The aggregator of the federated learning system is a base station or cloud server in the wireless communication network, and the client is various terminals in the wireless communication network, including smartphones, IoT sensors, and edge computing nodes. The aggregator configures the global model w0 and initializes the label average feature norm difference. Configure system hyperparameters in the aggregator; hyperparameters include the learning rate η. i Regularization parameter λ, total number of communication rounds M, number of clients I; Client i configures local model Feature extractor and local dataset D i Wherein, the superscript or subscript "i" represents client i; the subscript "t" represents the current communication round t, and the first round t = 0.
3. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 2, characterized in that: The aggregator broadcasts the global model parameters w for round t to all clients. t and characteristic norm difference In the first round of training, the global model parameters w in the aggregator... t and characteristic norm difference w0 and In subsequent rounds, the aggregator uses the average feature norm from all client feedback. Update feature norm difference Based on the model residuals reported by all clients Update global model parameters w t After the global model is updated, w will be t+1 and Distribute to clients.
4. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 3, characterized in that: The aggregator calculates the average feature norm based on feedback from all clients. Update feature norm difference The process is as follows: The aggregator calculates the feature norm difference for each label k. In the formula, This represents the average feature norm of the feedback from client m, excluding client i. The aggregator uses the model residuals from all clients as input. Update global model parameters w t The process is as follows: based on the transmission success probability p i For successfully uploaded model residuals Perform weighted aggregation and update the global model: In the formula, This represents the set of clients that successfully uploaded data in the current communication round t. For indicator functions, if Then α i =1, otherwise a i =0; β i Represents the weights of the model residuals for each client i; w t+1 These are the updated global model parameters; where the probability of successful transmission p i Obtained through historical channel statistics: In the formula, T represents the time scale.
5. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 3, characterized in that: The client receives the global model parameter w broadcast by the aggregator. t and characteristic norm difference For local model training, the client's local dataset originates from mobile devices, IoT devices, or edge computing nodes, and is data in the form of images, represented as follows: in, Labels representing local samples, For local samples, N i Let j represent the number of local samples in client i, and j represent the j-th local sample. The client-side feature extractor uses a convolutional neural network or a recurrent neural network, which is represented as follows: It will use local samples Extracting feature norms from it as input These are the parameters for the feature extractor. The client first extracts the feature norm using a feature extractor. Then, the average feature norm is calculated for each label k. Next, the difference in the received feature norms is combined Construct regularization terms Thus, a local training loss function is established. Train the local model to obtain the updated local model. Calculate the model residuals based on the updated local model. Together Upload to the aggregator.
6. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 5, characterized in that: The characteristic norm calculated by client i Represented as: Based on the characteristic norm Calculate the average feature norm for each label k. The process is as follows: in, For indicator functions, when At that time, α j ￠=1, otherwise α j ￠=0.
7. The wireless federated learning method based on feature norm regularization and statistical aggregation according to claim 5, characterized in that: Client i based on the received feature norm difference Constructed regularization terms for: in, Let K represent the proportion of label k in the dataset of client i, where K is the total number of labels in the dataset; The local training loss function Represented as: in, The cross-entropy loss is used, and λ is the feature norm regularization parameter. Therefore, gradient descent is used to update the local model, and the process is expressed as follows: Where, η i This represents the learning rate of client i. This represents the gradient operator.