Asynchronous federated image classification method and system based on client selection and aggregation weight adjustment

Abstract

The present invention relates to the field of image classification technology, and more particularly to an asynchronous federated image classification method and system based on client selection and aggregation weight adjustment. The method utilizes a federated learning algorithm to train an image classification model to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model and the similarity between local models. The image to be identified is input into the image classification target model, and the image classification target model is used to obtain and output the type of the image to be identified. The present invention can reduce the impact of data heterogeneity on the global model, mine valuable information from outdated local update models, enhance the utility of the global model, and improve the predictive performance of the global model during asynchronous federated image classification. The method is applicable to supervised data analysis scenarios including, but not limited to, image classification.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to an asynchronous federated image classification method and system based on client selection and aggregation weight adjustment. Background Art

[0002] Federated learning is a mainstream distributed learning method with privacy-preserving features. Its main feature is that a central server sends the global model to the client, while the central server aggregates the client's raw data. The basic idea is that the central server aggregates the local updated models uploaded by the clients to update the global model. Asynchronous federated learning means that all clients update their local models independently and in parallel. During the interaction with the central server, there is no need to wait for other clients, and the central server does not need to specify the set of clients participating in the update in advance. Asynchronous federated learning has the advantages of high utilization of local user computing resources, minimal impact on client status, and reduced overall training time.

[0003] In scenarios with significant heterogeneity in client computing resources, while asynchronous federated learning offers advantages such as high computational efficiency and robust aggregation, the effectiveness of the global model is impacted by client data heterogeneity and local update staleness. Data heterogeneity refers to the non-independent and identically distributed nature of the data, meaning that different clients have uneven image category labels. This can cause global model performance to favor more numerous categories during aggregation. Local update staleness refers to the fact that the locally updated model uploaded by the client is outdated relative to the current global model on the central server, causing this outdated information to undermine the effectiveness of the global model during aggregation.

[0004] Existing approaches to addressing the impact of client data heterogeneity in asynchronous federated learning primarily rely on selecting specific clients to participate in global model updates based on current data distribution information or update frequency. Approaches to addressing the impact of local update staleness primarily involve adjusting the client's aggregate weights based on staleness. However, client selection based on current data distribution information or update frequency only ensures the rationality of client selection at the current moment, not long-term rationality. Aggregate weight adjustment based on staleness simply suppresses the negative impact of outdated information while failing to leverage its positive impact. Therefore, leveraging long-term data distribution information to select clients and fully exploit the valuable information from outdated updates is crucial to further improve the predictive performance of asynchronous federated learning global models in image classification tasks. Summary of the Invention

[0005] To this end, the present invention provides an asynchronous federated image classification method and system based on client selection and aggregation weight adjustment, which solves the problem of poor global model utility in the asynchronous federated image classification process caused by the existing method of using current data distribution information and simply suppressing the influence of outdated information in asynchronous federated learning.

[0006] According to the design scheme provided by the present invention, on the one hand, an asynchronous federated image classification method based on client selection and aggregation weight adjustment is provided, comprising:

[0007] The image classification model is trained using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution.

[0008] The image to be identified is input into the image classification target model, and the image classification target model is used to obtain the type of the image to be identified and output it.

[0009] As the asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, the image classification model is further trained using a federated learning algorithm, including:

[0010] The central server initializes model parameters and a global image category label list, and broadcasts the initialized model parameters to all clients participating in the federated learning training, where all the clients participating in the federated learning training constitute a first client set. The global image category label list is used to record the cumulative distribution of client image data that has participated in the global model update;

[0011] Each client counts the local image data category label list and sends it to the central server;

[0012] Each client receives the global model parameters and uses them as local model parameters, randomly extracts a minimum batch of data from the local image data, trains the local model using the minimum batch data, and uploads the trained local model increment and the corresponding client to the cache queue of the central server;

[0013] The central server selects a specified number of clients from the cache queue according to the first-in-first-out principle, forms the selected clients into a second client set, selects target clients from the second client set based on the cumulative distribution of historical image data, updates the global image category label list, and forms the target clients into a third client set;

[0014] The central server adjusts the local update aggregation weight in the third client set based on the similarity between the local model and the global model and the similarity between the local models, and updates the current global model according to the aggregation weight and the target client in the third client set;

[0015] The central server sends the updated global model parameters to all clients in the second client set to iteratively execute the federated learning training process until the preset iteration conditions are met and the image classification target model is obtained.

[0016] As an asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, further, the central server selects a specified number of clients from the cache queue according to the first-in-first-out principle, including:

[0017] Set the required target client number K;

[0018] Based on the required number of target clients K and the preset amplification factor C af , select K*C from the cache queue according to the first-in-first-out principle af clients and form a second client set.

[0019] As the asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, client selection is further performed based on the cumulative distribution of historical image data, including:

[0020] Construct a client selection optimization model based on the historical cumulative distribution and the uniform distribution under the specified length under the specified client set;

[0021] The client selection optimization model is solved and the client set with the maximum cumulative distribution information entropy is selected.

[0022] As the asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, the client selection optimization model is further expressed as:

[0023]

[0024] in, Represents the target client set, is a uniform distribution of specified length, is the historical cumulative distribution of client image data, K is the required number of target clients, Specifies a client collection.

[0025] As an asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, further, adjusting the local update aggregation weight based on the similarity between the local model and the global model and the similarity between local models includes:

[0026] Use cosine similarity to calculate the similarity between the local model and the global model, and the similarity between local models. The similarity between the local model and the global model is the similarity between the local model increment of each target client and the global model, and the similarity between local models is the similarity between the local model increment of each target client and the local model increment of other target clients.

[0027] The similarity between the local model and the global model, as well as the similarity between local models, are aggregated based on the weight factor.

[0028] As the asynchronous federated image classification method based on client selection and aggregation weight adjustment of the present invention, further, the process of updating the current global model according to the aggregation weight and the target client in the third client set is expressed as follows:

[0029] Among them, w t is the current global model at the tth global model update, u i Set as target client The target client in C ds For weight adjustment based on staleness, is the normalized weight of the target client aggregate weight, The local model increment of the target client is outdated, and the outdatedness is the outdatedness of the local model increment uploaded by the target client when the current global model is updated.

[0030] On the other hand, the present invention also provides an asynchronous federated image classification system based on client selection and aggregation weight adjustment, comprising: a model training module and an image classification module, wherein:

[0031] A model training module is used to train the image classification model using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution.

[0032] The image classification module is used to input the image to be identified into the image classification target model, use the image classification target model to obtain the type of the image to be identified and output it.

[0033] Beneficial effects of the present invention:

[0034] The present invention selects clients based on the distribution of historical accumulated data, which can effectively reduce the impact of data heterogeneity on the global model, and adjusts the aggregation weight based on the similarity between the client's local update model and the central server's global model, and the similarity between local update models. It can mine valuable information from local outdated update models, improve the effectiveness of the global model, and improve the prediction performance of the global model in the asynchronous federated image classification process. It can be applied to supervised data analysis scenarios including but not limited to image classification, can effectively alleviate the negative impact of data heterogeneity and obsolescence, and significantly improve the prediction accuracy of the global model. It has good application prospects in the field of asynchronous federated image classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the asynchronous federated image classification process based on client selection and aggregation weight adjustment in an embodiment;

[0036] Figure 2 FIG. 2 is a diagram illustrating the federated training process in an embodiment. FIG. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below with reference to the accompanying drawings and technical solutions.

[0038] In view of the problems such as poor global model effectiveness in the existing asynchronous federated image classification process, the present invention embodiment, see Figure 1 As shown, an asynchronous federated image classification method based on client selection and aggregation weight adjustment is provided, including:

[0039] S101. Train an image classification model using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution.

[0040] S102: Input the image to be identified into the image classification target model, use the image classification target model to obtain the type of the image to be identified and output it.

[0041] During the federated training process, client selection is implemented based on the cumulative distribution information entropy of the data, which can effectively alleviate the impact of data heterogeneity. The similarity aggregation weight adjustment between the client's local update model and the global model, and between the local update models of different clients can alleviate the impact of obsolescence and improve the prediction accuracy of the global model for image classification.

[0042] Specifically, the federated learning algorithm is used to train the image classification model, which can be designed to include the following content:

[0043] The central server initializes model parameters and a global image category label list, and broadcasts the initialized model parameters to all clients participating in the federated learning training, where all the clients participating in the federated learning training constitute a first client set. The global image category label list is used to record the cumulative distribution of client image data that has participated in the global model update;

[0044] Each client counts the local image data category label list and sends it to the central server;

[0045] Each client receives the global model parameters and uses them as local model parameters, randomly extracts a minimum batch of data from the local image data, trains the local model using the minimum batch data, and uploads the trained local model increment and the corresponding client to the cache queue of the central server;

[0046] The central server selects a specified number of clients from the cache queue according to the first-in-first-out principle, forms the selected clients into a second client set, selects target clients from the second client set based on the cumulative distribution of historical image data, updates the global image category label list, and forms the target clients into a third client set;

[0047] The central server adjusts the local update aggregation weight in the third client set based on the similarity between the local model and the global model and the similarity between the local models, and updates the current global model according to the aggregation weight and the target client in the third client set;

[0048] The central server sends the updated global model parameters to all clients in the second client set to iteratively execute the federated learning training process until the preset iteration conditions are met and the image classification target model is obtained.

[0049] See also Figure 2 As shown in the figure, the process of federated training of image classification models can be summarized as follows:

[0050] Step 1: Model initialization. Central server Initialize the model parameter w0 and broadcast w0 to all participating customers. The set of all participating customers is recorded as

[0051] Step 2: Distribution Initialization. Central Server Initialize the global image label list Used to record the cumulative distribution of client image data that has participated in the global model update, where n c Indicates the number of image category labels.

[0052] Central Server Maintain a global list of image labels Any element p in this list i Indicates the proportion of the i-th category label in the total sample size of all data that have participated in the global model update, satisfying

[0053] Step 3: Client data distribution statistics. Each client Calculate the category label list for its local image data And send it to the central server in Indicates that the j-th image is on client u i The proportion of data that meets

[0054] Central Server Collect the distribution of client image labels at the beginning of training Used to update the global image label list in step 2 after learning begins The advantage of collecting the client image label distribution before training begins is that it avoids repeated collection of client distribution during each global aggregation after training starts, and also does not expose the private information of the number of images of each category in the client data.

[0055] Step 4: Client local training. Each client Randomly extract the smallest batch of size b from the local data to hold the latest global model Update. Re-recorded The update formula is:

[0056]

[0057] in Indicates that client u is updated at the tth global model update i Holds the global model version, E represents the number of local updates. After the local update is completed, the client will change the local model Upload to the central server

[0058] Step 5: Client selection. Central server Take out K*C from the cache queue according to the first-in-first-out principle af Local updates uploaded by clients, of which C af ≥1 represents the amplification factor, which is used to increase the information entropy of the cumulative distribution of the selected client data. The client set to be extracted is recorded as Its size is K*C afSolve the following optimization problem and select the user set that maximizes the cumulative distribution information entropy

[0059]

[0060] in The update formula is Indicates that The average distribution of user data in is added to the historical distribution The historical cumulative distribution after Indicates length n c Uniform distribution. Select the user set After that, update the global image label list

[0061] Central Server Based on historical cumulative distribution The benefit of selecting clients is to minimize the negative impact of data heterogeneity on the effectiveness of the global model. Specifically, it includes:

[0062] 1) Expand the scope of client selection. Central server Select K*C from the cache queue af clients, of which C af It is the amplification factor, and its purpose is to select clients that make the label distribution information entropy larger.

[0063] 2) Client selection. Central server according to Principle, select K clients. For a given length of distribution, the information entropy of uniform distribution is the largest. Therefore, this principle ensures that when adding client sets Finally, the information entropy of the cumulative distribution of all clients that have participated in the global update is maximized.

[0064] 3) Tag distribution update. Select a specific client set Then, using the client image label distribution collected in step 3 Update the global image label list in

[0065] Step 6: Aggregation weight adjustment. Central server Adjust the selected client set The aggregation weight is adjusted based on the similarity between the client's local updated model and the global model, and between the local updated models of different clients.

[0066] Specifically include:

[0067] 1) The similarity between the local updated model and the global model. The calculation formula is,

[0068]

[0069] 2) Similarity between local update and other client local update models. The calculation formula is,

[0070] 3) Comprehensive weight. Client u i The weight is

[0071] Based on global model information mining, the valuable information in the outdated local update model is used to improve the effectiveness of the global model. The calculation formula is, Ability to use the global model to mine valuable information in the local update model.

[0072] For the client The calculation formula is, The importance of information can be judged by using the similarity between client-side local update models.

[0073] Based on the weight factor, the global similarity and local similarity are integrated to ensure the rationality of the aggregation weight setting.

[0074] Step 7: Global model update. Central server Based on user collection and aggregation weight Update the current global model w as follows t :

[0075]

[0076] in for The normalized weight of Based on obsolescence The weight adjustment, Indicates that client u is updated at the current tth global model update i Staleness of uploaded local updates.

[0077] Central Server For client u i Weight Adjust to obtain polymerization in Taking into account the impact of outdatedness, normalized weights The impact of the number of clients is taken into account. When updating the global model, both the impact of outdatedness and the number of clients are considered to improve the effectiveness of the global model.

[0078] Step 8: Global model feedback. Central server The updated global model w t+1 Send to user collection Repeat steps 4 to 7 until the preset stop condition is reached.

[0079] In the above federated training process, the client selection based on the historical accumulated data distribution is used to reduce the impact of data heterogeneity on the global model. The aggregation weight adjustment based on the similarity between the client's local update model and the central server's global model and the similarity between local update models is used to mine valuable information from the local outdated update model and improve the effectiveness of the global model. This makes the trained target model applicable to supervised data analysis scenarios including but not limited to image classification, effectively alleviating the negative impact of data heterogeneity and obsolescence, and significantly improving the prediction accuracy of the global model.

[0080] Furthermore, based on the above method, an embodiment of the present invention also provides an asynchronous federated image classification system based on client selection and aggregation weight adjustment, comprising: a model training module and an image classification module, wherein:

[0081] A model training module is used to train the image classification model using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution.

[0082] The image classification module is used to input the image to be identified into the image classification target model, use the image classification target model to obtain the type of the image to be identified and output it.

[0083] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0085] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0086] Those skilled in the art will appreciate that all or part of the steps in the above method can be performed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps in the above embodiment can be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or software functional modules. The present invention is not limited to any specific combination of hardware and software.

[0087] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. An asynchronous federated image classification method based on client selection and aggregation weight adjustment, characterized in that: Include: The image classification model is trained using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution. Among them, the federated learning algorithm is used to train the image classification model, including: The central server initializes model parameters and a global image category label list, and broadcasts the initialized model parameters to all clients participating in the federated learning training, where all the clients participating in the federated learning training constitute a first client set. The global image category label list is used to record the cumulative distribution of client image data that has participated in the global model update; Each client counts the local image data category label list and sends it to the central server; Each client receives the global model parameters and uses them as local model parameters, randomly extracts a minimum batch of data from the local image data, trains the local model using the minimum batch data, and uploads the trained local model increment and the corresponding client to the cache queue of the central server; The central server selects a specified number of clients from the cache queue according to the first-in-first-out principle, forms the selected clients into a second client set, selects target clients from the second client set based on the cumulative distribution of historical image data, updates the global image category label list, and forms the target clients into a third client set; The central server adjusts the local update aggregation weight in the third client set based on the similarity between the local model and the global model and the similarity between the local models, and updates the current global model according to the aggregation weight and the target client in the third client set; The central server sends the updated global model parameters to all clients in the second client set to iteratively execute the federated learning training process until the preset iteration conditions are met and the image classification target model is obtained; The process of updating the current global model based on the aggregation weight and the target client in the third client set is expressed as: Among them, w t is the current global model at the tth global model update, u i Set as target client The target client in C ds For weight adjustment based on staleness, is the normalized weight of the target client aggregate weight, The local model increment of the target client, and the outdatedness is the outdatedness of the local model increment uploaded by the target client when the current global model is updated; The image to be identified is input into the image classification target model, and the image classification target model is used to obtain the type of the image to be identified and output it.

2. The asynchronous federated image classification method based on client selection and aggregation weight adjustment according to claim 1 is characterized in that: The central server selects a specified number of clients from the cache queue based on the first-in-first-out principle, including: Set the required target client number K; Based on the required number of target clients K and the preset amplification factor C af , select K*C from the cache queue according to the first-in-first-out principle af clients and form a second client set.

3. The asynchronous federated image classification method based on client selection and aggregation weight adjustment according to claim 1 is characterized in that: Client selection based on the cumulative distribution of historical image data, including: Construct a client selection optimization model based on the historical cumulative distribution and the uniform distribution under the specified length under the specified client set; The client selection optimization model is solved and the client set with the maximum cumulative distribution information entropy is selected.

4. The asynchronous federated image classification method based on client selection and aggregation weight adjustment according to claim 3 is characterized in that: The client selection optimization model is expressed as: in, Represents the target client set, is a uniform distribution of specified length, is the historical cumulative distribution of client image data, K is the required number of target clients, Specifies a client collection.

5. The asynchronous federated image classification method based on client selection and aggregation weight adjustment according to claim 1 is characterized in that: Adjust the local update aggregation weight based on the similarity between the local model and the global model and the similarity between local models, including: Use cosine similarity to calculate the similarity between the local model and the global model, and the similarity between local models. The similarity between the local model and the global model is the similarity between the local model increment of each target client and the global model, and the similarity between local models is the similarity between the local model increment of each target client and the local model increment of other target clients. The similarity between the local model and the global model, as well as the similarity between local models, are aggregated based on the weight factor.

6. An asynchronous federated image classification system based on client selection and aggregation weight adjustment, characterized in that: Contains: model training module and image classification module, among which, A model training module is used to train the image classification model using a federated learning algorithm to obtain an image classification target model. The federated learning algorithm selects target clients based on the cumulative distribution of historical image data and adjusts aggregation weights based on the similarity between the local model and the global model, and the similarity between local models. The data distribution is the proportion of each type of image data in the corresponding image data, and the cumulative distribution of historical image data is obtained by adding the user's local image data distribution to the historical global image data distribution. Among them, the federated learning algorithm is used to train the image classification model, including: The central server initializes model parameters and a global image category label list, and broadcasts the initialized model parameters to all clients participating in the federated learning training, where all the clients participating in the federated learning training constitute a first client set. The global image category label list is used to record the cumulative distribution of client image data that has participated in the global model update; Each client counts the local image data category label list and sends it to the central server; Each client receives the global model parameters and uses them as local model parameters, randomly extracts a minimum batch of data from the local image data, trains the local model using the minimum batch data, and uploads the trained local model increment and the corresponding client to the cache queue of the central server; The central server selects a specified number of clients from the cache queue according to the first-in-first-out principle, forms the selected clients into a second client set, selects target clients from the second client set based on the cumulative distribution of historical image data, updates the global image category label list, and forms the target clients into a third client set; The central server adjusts the local update aggregation weight in the third client set based on the similarity between the local model and the global model and the similarity between the local models, and updates the current global model according to the aggregation weight and the target client in the third client set; The central server sends the updated global model parameters to all clients in the second client set to iteratively execute the federated learning training process until the preset iteration conditions are met and the image classification target model is obtained; The process of updating the current global model based on the aggregation weight and the target client in the third client set is expressed as: Among them, w t is the current global model at the tth global model update, u i Set as target client The target client in C ds For weight adjustment based on staleness, is the normalized weight of the target client aggregate weight, The local model increment of the target client, and the outdatedness is the outdatedness of the local model increment uploaded by the target client when the current global model is updated; The image classification module is used to input the image to be identified into the image classification target model, use the image classification target model to obtain the type of the image to be identified and output it.

7. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 5 can be implemented.

Images