A Semantic Federated Learning Optimization Method Based on Data Quality

Abstract

This invention relates to the field of federated learning technology and discloses a semantic federated learning optimization method based on data quality. The method is characterized by a system composed of clients, each client corresponding to a local dataset of size ρ. The advantages of this invention are: it considers factors affecting the staleness of client data, uses data quality indicators as key indicators affecting federated learning performance, and fully considers data volume and data freshness, making the learning performance closer to reality. It also considers system energy consumption, rationally selects the optimal client combination to participate in aggregation, and effectively solves the data staleness problem in semantic federated learning. This invention uses a client selection set expansion algorithm, based on ρ... n The client values ​​are sorted in ascending order, and the best client in this round is gradually added to the selection set to obtain the optimal client set. This solves the problem of data staleness while taking into account the optimal allocation of bandwidth.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, specifically to a semantic federated learning optimization method based on data quality. Background Technology

[0002] In traditional computing paradigms, user devices typically upload raw data to centralized cloud servers for processing, but this inevitably leads to significant transmission overhead and data privacy leaks. To address this issue, federated learning solutions have gradually become a research focus. As a distributed machine learning paradigm, federated learning aggregates locally trained models into a globally shared model, avoiding the drawbacks of privacy data leaks and facilitating a fast and efficient training process.

[0003] Semantic federated learning is an enhanced federated learning method that aims to improve model performance by combining semantic information from distributed data sources. Due to differences in client data volume and device processing speed, traditional federated averaging algorithms result in long waiting times for faster clients, leading to data staleness issues. Many current studies address this problem by using asynchronous aggregation. However, asynchronous updates can cause inconsistencies in the model's state across different participants, potentially impacting the final model's performance. Furthermore, designing and implementing asynchronous federated learning algorithms is generally more complex than synchronous algorithms, requiring significantly more computational resources. In addition, while some studies consider data freshness as a metric for asynchronous aggregation decisions, this overlooks other performance metrics affecting federated learning, such as the amount of original data.

[0004] Therefore, designing an asynchronous semantic federated learning method that can solve the problem of data staleness is an urgent problem to be solved. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a semantic federated learning optimization method based on data quality. This method has advantages such as rationally selecting the optimal combination of clients to participate in aggregation, and solves the problem of data staleness in semantic federated learning.

[0007] (II) Technical Solution

[0008] To address the data staleness problem in semantic federated learning, this invention provides the following technical solution:

[0009] A semantic federated learning optimization method based on data quality, characterized in that the system consists of... It consists of 1 client, each client The corresponding local dataset is Dataset size is Client n uses local dataset Training a local model To minimize the loss function F n (g); All clients collaborate to train a global model g (t) To minimize the global loss, This indicates whether client n was selected in round t. The optimization method specifically includes the following steps:

[0010] Step S1: The server distributes the global model g. (t) ;

[0011] Step S2: After receiving the global model, the client uses the corresponding algorithm on the local dataset. The local model is obtained by training on it.

[0012] Step S3: Use data quality Q n As a metric for measuring the performance U of federated learning;

[0013] Step S4: Select the optimal client combination to participate in aggregation based on the impact of energy consumption and bandwidth allocation factors on federated learning performance;

[0014] Step S5: Upload the selected client model parameters to the server for aggregation to obtain a new round of global models;

[0015] Complete the global model and iterate the above process until the model converges or the set conditions are met.

[0016] Preferably, in step S2, the client obtains the local model:

[0017] S2.1 First, the client receives the global model g sent by the server. (t) The structure of the global model is determined by the server;

[0018] S2.2 Then the client uses the appropriate algorithm (such as gradient descent) on the local data. The local model is obtained by updating the received global model parameters.

[0019] Preferably, the data quality Q used in step S3 is... n As a metric for measuring the performance U of federated learning:

[0020] S3.1 First, data quality Where A n This represents the Information Age (AOI) value of client n.

[0021] S3.2 The performance of federated learning is then expressed as in η t It is a time weight, used to represent the importance of selecting more client aggregations in different learning rounds.

[0022] Preferably, the impact of energy consumption and bandwidth allocation on federated learning performance in step S4 selects the clients that need to participate in the aggregation.

[0023] S4.1 First, for the client n selected in the t-th round, the training energy consumption is... Transmission energy consumption is expressed as: in Indicates bandwidth allocation rate. Indicates the channel state;

[0024] The total energy consumption of client n in S4.2 is expressed as:

[0025]

[0026] S4.3 utilizes the Lyapunov technique to construct a virtual energy defect queue q for each client n. n (t) to guide client selection and bandwidth allocation decisions, q n (t) is represented as:

[0027]

[0028] S4.4 requires maximizing the difference between federated learning performance and energy consumption, i.e.:

[0029]

[0030] S4.5 Selects the client with the best bandwidth allocation in this round based on federated learning performance and adds it to the selection set S, with the initial value of the selection set being S. 0 ;

[0031] S4.6 According to parameters The values ​​are added to the selection set S in ascending order.

[0032] S4.7 uses b * (S) represents the optimal bandwidth allocation for the selection set S, O * (S) represents the optimal value, O * (S 0 )=η t |S 0 |;

[0033] S4.8 uses an algorithm to sequentially solve for clients n∈SS that satisfy the conditions. 0 ;

[0034] S4.8 The selection set S contains all possible sets. The optimal bandwidth allocation and client selection are solved in the middle.

[0035]

[0036] b * =b * (S * ).

[0037] Preferably, the selected client model parameters are uploaded to the server for aggregation to obtain a new round of global models:

[0038] S5.1 The optimal choice set S obtained according to claim 4 * , obtain the client that needs to be uploaded;

[0039] The S5.2 server uploads and aggregates the model parameters from the clients that need to be uploaded, and after aggregation, a new global model g is obtained. (t+1) Then, the global model is sent to the client.

[0040] (III) Beneficial Effects

[0041] Compared with existing technologies, this invention provides a semantic federated learning optimization method based on data quality, which has the following beneficial effects:

[0042] This paper presents a data quality-based optimization method for semantic federated learning. This method considers factors affecting the staleness of client data, uses data quality metrics as key indicators influencing federated learning performance, and fully considers data volume and information freshness to make learning performance more closely resemble real-world conditions. It also considers system energy consumption and rationally selects the optimal combination of clients to participate in aggregation, effectively solving the data staleness problem in semantic federated learning.

[0043] This invention uses a client-selected set expansion algorithm, based on ρ n The client values ​​are sorted in ascending order, and the best client in this round is gradually added to the selection set to obtain the optimal client set. This solves the problem of data staleness while taking into account the optimal allocation of bandwidth. Attached Figure Description

[0044] Figure 1 This is a flowchart of the optimization method of the present invention;

[0045] Figure 2 A flowchart of the client selection set expansion algorithm for this invention. Detailed Implementation

[0046] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] A semantic federated learning optimization method based on data quality includes:

[0048] First, the client receives the global model g sent by the server. (t) The structure of the global model is determined by the server.

[0049] Then the client uses the appropriate algorithm (such as gradient descent) on the local data. The local model is obtained by updating the received global model parameters.

[0050] Further use of data quality Q n As a metric for measuring the performance of federated learning, U.

[0051] First, data quality Where A n This represents the Information Age (AOI) value of client n.

[0052] Then the performance of federated learning is expressed as in η t It is a time weight, used to represent the importance of selecting more client aggregations in different learning rounds.

[0053] Further consideration is given to the impact of energy consumption and bandwidth allocation on federated learning performance to select clients that need to participate in aggregation.

[0054] Firstly, for the client n selected in round t, the training energy consumption is... Transmission energy consumption is expressed as in Indicates bandwidth allocation rate. Indicates the channel status.

[0055] Then the total energy consumption of client n is expressed as:

[0056]

[0057] Then, using the Lyapunov technique, a virtual energy defect queue q is constructed for each client n. n (t) to guide client selection and bandwidth allocation decisions, q n (t) is represented as:

[0058]

[0059] The next step is to maximize the difference between federated learning performance and energy consumption, namely:

[0060]

[0061] Subsequently, based on the performance of federated learning, the client with the best bandwidth allocation in this round is selected and added to the selection set S, with the initial value of the selection set being S. 0 .

[0062] Then based on the parameters The values ​​are used to add clients n to the selection set S in ascending order.

[0063] Use b * (S) represents the optimal bandwidth allocation for the selection set S, O * (S) represents the optimal value, O * (S 0 )=η t |S 0 |

[0064] The algorithm is used to sequentially solve for clients n∈SS that satisfy the conditions. 0 .

[0065] Ultimately, the selection set S is the set of all possible choices. The optimal bandwidth allocation and client selection are solved in the middle.

[0066]

[0067] b * =b * (S * ).

[0068] The selected client model parameters are then uploaded to the server and aggregated to obtain a new round of global models.

[0069] Based on the optimal choice set S obtained in the previous step * , and obtain the client that needs to be uploaded.

[0070] The server uploads and aggregates the model parameters from the clients that need to be uploaded, and after aggregation, a new global model g is obtained. (t+1) Then, the global model is sent to the client.

[0071] A semantic federated learning optimization method based on data quality includes the following steps:

[0072] Step 1: System Initialization

[0073] (1) Initialize system components and set up the server and client.

[0074] (2) Model initialization: The server initializes global model parameters.

[0075] Step 2: Local Model Training

[0076] (1) Data partitioning: dividing the client's local dataset Divided into small batches.

[0077] (2) The client uses small batches of data to train the local model and update the local model parameters.

[0078] Step 3: Defining Data Quality

[0079] Data quality metrics are defined using local data volume and data freshness.

[0080] Step 4: Client Selection

[0081] (1) Using the data quality metric Q n Learning performance U is defined as the primary influencing factor.

[0082] (2) Use the difference between maximum learning performance and energy consumption to solve for the optimal selection set in this round.

[0083] (3) Select a subset of energy-efficient clients to participate in global aggregation based on the final set of the selection set.

[0084] Step 5: Model Update and Aggregation

[0085] (1) Model update: The client sends the updated local model parameters to the server.

[0086] (2) The server aggregates all client model parameters selected in step four into global model parameters.

[0087] Step 6: Iterative Training

[0088] The client continues to train locally using the updated global model parameters, repeating steps two through five until the termination condition is met.

[0089] The beneficial effects of this invention are: a semantic federated learning optimization method based on data quality. This method considers factors affecting the staleness of client data, uses data quality indicators as key indicators affecting federated learning performance, and fully considers factors such as data volume and data freshness, making the learning performance closer to reality. It also considers system energy consumption, rationally selects the optimal client combination to participate in aggregation, and effectively solves the data staleness problem in semantic federated learning.

[0090] This invention uses a client-selected set expansion algorithm, based on ρ n The client selection set is sorted in ascending order, and the best client in each round is gradually added to the selection set to obtain the optimal client set. This approach addresses data staleness while considering optimal bandwidth allocation.

[0091] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A semantic federated learning optimization method based on data quality, characterized in that, The system consists of It consists of 1 client, each client The corresponding local dataset is Dataset size is ; Client Using local datasets Training a local model To minimize the loss function ; All clients collaborate to train a global model. To minimize the global loss, Indicates the client in round t Whether it is selected or not, the optimization method specifically includes the following steps: Step S1: The server distributes the global model. ; Step S2: After receiving the global model, the client uses the corresponding algorithm on the local dataset. The local model is obtained by training on it. ; Step S3: Use data quality As a measure of federated learning performance Indicators; Step S4: Select the optimal client combination to participate in aggregation based on the impact of energy consumption and bandwidth allocation factors on federated learning performance; Step S5: Upload the selected client model parameters to the server for aggregation to obtain a new round of global models; Complete the global model and iterate the above process until the model converges or the set conditions are met. Step S3 describes the use of data quality As a measure of federated learning performance Indicators; S3.1 First, data quality ,in Indicates client Information Age (AOI) value; S3.2 The performance of federated learning is then expressed as ,in , It is a time weight, used to represent the importance of selecting more client aggregations in different learning rounds; The impact of energy consumption and bandwidth allocation on federated learning performance described in step S4 selects the clients that need to participate in aggregation; S4.1 First, select the client in round t. Training energy consumption is Transmission energy consumption is expressed as ,in Indicates bandwidth allocation rate. The channel state is represented by τ̄, which represents the preset transmission duration for uploading the client model in each round of federated learning; N_0 represents the noise power spectral density in the uplink communication channel; B represents the total uplink bandwidth available to all clients in the system; and S4.2 client... The total energy consumption is expressed as: ， S4.3 utilizes Lyapunov technology for each client Construct a virtual energy defect queue To guide client selection and bandwidth allocation decisions. Represented as: ； Hn represents the total energy budget that client n is allowed to consume throughout the entire federated learning cycle; T represents the preset total number of federated learning rounds, so Hn / T represents the average allowed energy consumption of client n in each round of federated learning; S4.4 requires maximizing the difference between federated learning performance and energy consumption, i.e.: ； Here, V>0 represents the trade-off control parameter in Lyapunov optimization, used to adjust the relative importance between federated learning performance and client energy constraints. A larger V value makes client selection and bandwidth allocation more focused on improving federated learning performance, while a smaller V value makes the optimization process more focused on controlling client energy consumption and maintaining the stability of the virtual energy deficit queue. S4.5 selects the client with the best bandwidth allocation in this round based on federated learning performance and adds it to the selection set. The initial value of the selection set is , S4.6 According to parameters Values, sorted in ascending order for the client Add to selection set one by one middle; S4.7 usage Represents the selection set Optimal bandwidth allocation. Indicates optimality. ; S4.8 uses an algorithm to sequentially solve for clients that meet the conditions. ; S4.8 in the selection set All possible sets The optimal bandwidth allocation and client selection are solved in the middle. ; 2. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The client obtains the local model as described in step S2. S2.1 First, the client receives the global model sent by the server. The structure of the global model is determined by the server; S2.2 Then the client uses the corresponding algorithm to process the local data. The local model is obtained by updating the received global model parameters. .

3. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The selected client-side model parameters are uploaded to the server for aggregation to obtain a new round of global models. S5.1 Based on the optimal choice set obtained in S4 , obtain the client that needs to be uploaded; The S5.2 server uploads and aggregates the model parameters from the clients that need to be uploaded, and obtains a new global model after the aggregation is complete. Then, the global model is sent to the client.

Images