Semantic federal learning optimization method based on data quality

Abstract

The invention relates to the technical field of federated learning, and discloses a semantic federated learning optimization method based on data quality, which is characterized in that a system is composed of # imgabs0 clients, and a local data set corresponding to # imgabs1 # of each client is # imgabs2 # of which the size is # imgabs3 #. A data quality index is used as a key index influencing federal learning performance, and data volume and data information freshness factors are fully considered, so that the learning performance is closer to the actual situation. According to the method, a client selection set expansion algorithm is used, clients are arranged in an ascending order according to a rhan value, the optimal clients in the round are gradually added into a selection set, and the optimal client combination needing to participate in aggregation is reasonably selected by considering the energy consumption of a system, so that the problem of data obsolessness of semantic federation learning is effectively solved. According to the method, the optimal bandwidth allocation is considered while the problem of data obsolessness is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of federated learning, and in particular to a semantic federated learning optimization method based on data quality. Background Art

[0002] In the traditional computing paradigm, user devices usually upload raw data to a centralized cloud server for processing, but this inevitably causes huge transmission overhead and data privacy leakage. In response to this problem, federated learning solutions have gradually become a research focus. As a distributed machine learning paradigm, federated learning uses localized model training results to aggregate into a global shared model, avoiding the disadvantages of privacy data leakage and facilitating a fast and efficient training process.

[0003] Semantic federated learning is an enhanced federated learning method that aims to improve the performance of the model by combining the semantic information of distributed data sources. Due to the differences in client data volume and device processing speed, the traditional federated averaging algorithm causes clients with fast training speed to wait for a long time, resulting in data staleness. Currently, many studies use asynchronous aggregation to solve the problem of data staleness. However, asynchronous updates may cause inconsistent states of the model between different participants, and this inconsistency may affect the performance of the final model. At the same time, the design and implementation of asynchronous federated learning algorithms are usually more complex than synchronous algorithms and require more computing resources. In addition, although some studies have considered the freshness of data information and used information freshness as an indicator for asynchronous aggregation decisions, this ignores other indicators that affect the performance of federated learning, such as the amount of original data.

[0004] Therefore, how to design an asynchronous semantic federated learning method that can solve the problem of data staleness is an urgent problem to be solved. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a semantic federated learning optimization method based on data quality, which has the advantages of reasonably selecting the optimal client combination that needs to participate in aggregation, and solves the problem of data staleness in semantic federated learning.

[0007] (II) Technical solution

[0008] In order to solve the data staleness problem of semantic federated learning, the present invention provides the following technical solutions:

[0009] A semantic federated learning optimization method based on data quality, characterized in that the system consists of clients, each client The corresponding local dataset is The dataset size is Client n uses a local dataset Training a local model To minimize the loss function F n (g); All clients collaborate to train a global model g (t) To minimize the global loss, Indicates whether client n is selected in round t. The optimization method specifically includes the following steps:

[0010] Step S1: The server sends the global model g (t) ;

[0011] Step S2: After receiving the global model, the client uses the corresponding algorithm to calculate the local data set Train on it to get the local model

[0012] Step S3: Use data quality Q n As an indicator to measure the performance U of federated learning;

[0013] Step S4: Select the optimal client combination participating in the aggregation according to the impact of energy consumption and bandwidth allocation factors on the performance of federated learning;

[0014] Step S5: uploading the selected client model parameters to the server to aggregate and obtain a new round of global model;

[0015] Complete the global model and continuously iterate the above process until the model converges or meets the set conditions.

[0016] Preferably, the client in step S2 obtains a local model:

[0017] S2.1 First, the client receives the global model g sent by the current server (t) ,The structure of the global model is determined by the server;

[0018] S2.2 The client then uses the corresponding algorithm (such as gradient descent algorithm) to calculate the local data The received global model parameters are updated to obtain the local model

[0019] Preferably, the quality of the data used in step S3 is Q n As an indicator to measure the performance U of federated learning:

[0020] S3.1 First, data quality Among them A n Indicates the age of information (AOI) value of client n;

[0021] S3.2 The performance of federated learning is then expressed as in η t is the time weight, which is used to indicate the importance of selecting more client aggregations in different learning rounds.

[0022] Preferably, the impact of energy consumption and bandwidth allocation on the performance of federated learning in step S4 selects the clients that need to participate in the aggregation.

[0023] S4.1 first trains the client n selected in round t with the energy consumption The transmission energy consumption is expressed as: in Indicates the bandwidth allocation rate, Indicates the channel status;

[0024] The total energy consumption of S4.2 client n is expressed as:

[0025]

[0026] S4.3 Use Lyapunov technology to construct a virtual energy defect queue q for each client n n (t), to guide client selection and bandwidth allocation decisions, q n (t) is expressed as:

[0027]

[0028] S4.4 needs to maximize the difference between federated learning performance and energy consumption, that is:

[0029]

[0030] S4.5 selects the client with the best bandwidth allocation in this round according to the federated learning performance and adds it to the selection set S. The initial value of the selection set is S 0 ;

[0031] S4.6 According to the parameters Value, add client n to the selection set S in ascending order;

[0032] S4.7 Use b * (S) represents the optimal bandwidth allocation of the selection set S, O * (S) indicates optimal, O * (S 0 ) = η t |S 0 |;

[0033] S4.8 Use the algorithm to solve the client n∈SS that meets the conditions in turn 0 ;

[0034] S4.8 All possible sets in the selection set S Solve for optimal bandwidth allocation and client selection

[0035]

[0036] b * =b * (S * ).

[0037] Preferably, the selected client model parameters are uploaded to the server for aggregation to obtain a new round of global model:

[0038] S5.1 The optimal selection set S obtained according to claim 4 * , get the client that needs to upload;

[0039] S5.2 The server aggregates the model parameters of the client that need to be uploaded, and obtains the new global model g after the aggregation is completed. (t+1) , and then send the global model to the client.

[0040] (III) Beneficial effects

[0041] Compared with the prior art, the present invention provides a semantic federated learning optimization method based on data quality, which has the following beneficial effects:

[0042] This is a semantic federated learning optimization method based on data quality. This method takes into account the factors that affect the staleness of client data, uses data quality indicators as key indicators that affect the performance of federated learning, and fully considers the factors of data volume and data information freshness, making the learning performance closer to the actual situation. At the same time, the energy consumption of the system is considered, and the optimal client combination that needs to participate in the aggregation is reasonably selected, which effectively solves the data staleness problem of semantic federated learning.

[0043] The present invention uses the client selection set expansion algorithm, according to p n The clients are arranged in ascending order, and the optimal clients of this round are gradually added to the selection set to obtain the optimal client set, which not only solves the problem of data staleness but also takes into account the optimal allocation of bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the optimization method of the present invention;

[0045] Figure 2 The client selection set expansion algorithm block diagram of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] A semantic federated learning optimization method based on data quality, comprising:

[0048] First, the client receives the global model g sent by the current server (t) ,The structure of the global model is determined by the server.

[0049] The client then uses the corresponding algorithm (such as gradient descent algorithm) to calculate the local data The received global model parameters are updated to obtain the local model

[0050] Further use of data quality Q n As an indicator to measure the performance U of federated learning.

[0051] First, data quality Among them A n Indicates the Age of Information (AOI) value of client n.

[0052] Then the performance of federated learning is expressed as in η t is the time weight, which is used to indicate the importance of selecting more client aggregations in different learning rounds.

[0053] Further considering the impact of energy consumption and bandwidth allocation on the performance of federated learning, the clients that need to participate in the aggregation are selected.

[0054] First, for the client n selected in round t, the training energy consumption is The transmission energy consumption is expressed as in Indicates the bandwidth allocation rate, Indicates the channel status.

[0055] The total energy consumption of client n is then expressed as:

[0056]

[0057] Then, a virtual energy defect queue q is constructed for each client n using the Lyapunov technique. n (t), to guide client selection and bandwidth allocation decisions, q n (t) is expressed as:

[0058]

[0059] Next, we need to maximize the difference between federated learning performance and energy consumption, namely:

[0060]

[0061] Then, based on the federated learning performance, the client with the best bandwidth allocation in this round is selected and added to the selection set S. The initial value of the selection set is S 0 .

[0062] Then according to the parameters Value, add client n to the selection set S in ascending order.

[0063] Use b * (S) represents the optimal bandwidth allocation of the selection set S, O * (S) indicates optimal, O * (S 0 ) = η t |S 0 |.

[0064] Use the algorithm to solve the clients n∈SS that meet the conditions in turn 0 .

[0065] Finally, in the selection set S, all possible sets Solve for optimal bandwidth allocation and client selection

[0066]

[0067] b * =b * (S * ).

[0068] The selected client model parameters are further uploaded to the server for aggregation to obtain a new round of global model.

[0069] According to the optimal selection set S obtained in the previous step * , get the client that needs to upload.

[0070] The server aggregates the model parameters of the client that need to be uploaded, and after the aggregation is completed, a new global model g is obtained (t+1) , and then send the global model to the client.

[0071] A semantic federated learning optimization method based on data quality includes the following processes:

[0072] Step 1: System Initialization

[0073] (1) Initialize system components and set up the server and client.

[0074] (2) Model initialization: The server initializes the global model parameters.

[0075] Step 2: Local model training

[0076] (1) Data partitioning: dividing the client's local data set into Divide into small batches.

[0077] (2) The client uses small batches of data to train the local model and update the local model parameters.

[0078] Step 3: Data Quality Definition

[0079] Defining data quality indicators using local data volume and data information freshness

[0080] Step 4: Client selection

[0081] (1) Use data quality indicator Q n The learning performance U is defined as the main influencing factor.

[0082] (2) Use the difference between the maximum learning performance and the energy consumption to solve the optimal selection set for this round.

[0083] (3) According to the final set of the selection set, some clients with optimal energy consumption are selected to participate in the global aggregation.

[0084] Step 5: Model Update and Aggregation

[0085] (1) Model update: the client sends the updated local model parameters to the server.

[0086] (2) The server aggregates all client model parameters selected in the fourth step into global model parameters.

[0087] Step 6: Iterative training

[0088] The client continues to use the updated global model parameters for local training, repeating steps 2 to 5 until the termination condition is met.

[0089] The beneficial effects of the present invention are: a semantic federated learning optimization method based on data quality, which takes into account factors affecting the staleness of client data, uses data quality indicators as key indicators affecting the performance of federated learning, and fully considers factors such as data volume and data information freshness, so that the learning performance is closer to the actual situation. At the same time, the energy consumption of the system is taken into account, and the optimal client combination that needs to participate in the aggregation is reasonably selected, which effectively solves the data staleness problem of semantic federated learning.

[0090] The present invention uses the client selection set expansion algorithm, according to p n The client is sorted in ascending order, and the optimal client of this round is gradually added to the selection set to obtain the optimal client set, which not only solves the problem of data staleness but also considers the optimal allocation of bandwidth.

[0091] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A semantic federated learning optimization method based on data quality, characterized in that: The system consists of clients, each of which The corresponding local dataset is The dataset size is The client n uses the local dataset Training a local model To minimize the loss function F n (g) All the clients collaborate to train a global model g (t) To minimize the global loss, Indicates whether client n is selected in round t. The optimization method specifically includes the following steps: Step S1: The server sends the global model g (t) ; Step S2: After receiving the global model, the client uses the corresponding algorithm to calculate the local data set Train on it to get the local model Step S3: Use data quality Q n As an indicator to measure the performance U of federated learning; Step S4: Select the optimal client combination participating in the aggregation according to the impact of energy consumption and bandwidth allocation factors on the performance of federated learning; Step S5: uploading the selected client model parameters to the server to aggregate and obtain a new round of global model; Complete the global model and continuously iterate the above process until the model converges or meets the set conditions.

2. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The client in step S2 obtains the local model: S2.1 First, the client receives the global model g sent by the current server (t) ,The structure of the global model is determined by the server; S2.2 The client then uses the corresponding algorithm (such as gradient descent algorithm) to calculate the local data The received global model parameters are updated to obtain the local model 3. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The quality Q of the used data in step S3 n As an indicator to measure the performance U of federated learning: S3.1 First, data quality Among them A n Indicates the age of information (AOI) value of client n; S3.2 The performance of federated learning is then expressed as in η t is the time weight, which is used to indicate the importance of selecting more client aggregations in different learning rounds.

4. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The impact of energy consumption and bandwidth allocation on the performance of federated learning in step S4 selects the clients that need to participate in the aggregation. S4.1 first trains the client n selected in round t with the energy consumption The transmission energy consumption is expressed as in Indicates the bandwidth allocation rate, Indicates the channel status; S4.2 The total energy consumption of client n is expressed as: S4.3 Use Lyapunov technology to construct a virtual energy defect queue q for each client n n (t), to guide client selection and bandwidth allocation decisions, q n (t) is expressed as: S4.4 needs to maximize the difference between federated learning performance and energy consumption, that is: S4.5 selects the client with the best bandwidth allocation in this round according to the federated learning performance and adds it to the selection set S. The initial value of the selection set is S 0 ; S4.6 According to the parameters Value, add client n to the selection set S in ascending order; S4.7 Use b * (S) represents the optimal bandwidth allocation of the selection set S, O * (S) indicates optimal, O * (S 0 ) = η t |S 0 |; S4.8 Use the algorithm to solve the client n∈SS that meets the conditions in turn 0 ; S4.8 All possible sets in the selection set S Solve for optimal bandwidth allocation and client selection b * =b * (S * )。 5. The semantic federated learning optimization method based on data quality according to claim 1, characterized in that: The selected client model parameters are uploaded to the server and aggregated to obtain a new round of global models: S5.1 The optimal selection set S obtained according to claim 4 * , get the client that needs to upload; S5.2 The server aggregates the model parameters of the client that need to be uploaded, and obtains the new global model g after the aggregation is completed. (t+1) , and then send the global model to the client.

Images