A Clustering-Based Personalized Federated Learning Image Classification Method and System

Abstract

The present invention relates to artificial intelligence technology, and discloses a clustering-based personalized federated learning image classification method and system. The method includes: a client generates local model parameters and uploads them to a server; the server receives the local model parameters and performs complete clustering, and when the client updates the local model parameters, the server performs incremental clustering operations to obtain the final clustering parameter clusters; calculates the semi-global model parameters of each parameter cluster and calculates the client scores to select proxy clients, and the server sends the semi-global model parameters and the global model parameters to the proxy clients; the proxy clients perform local adaptive aggregation operations, and after the aggregation is completed, a personalized image classification model is obtained. The present invention also proposes an apparatus and a system for a clustering-based personalized federated learning image classification method. The present invention can solve the non-independent and identically distributed problem in the medical image data of each client and enhance the personalized expression of the local models within each client.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to a clustering-based personalized federated learning image classification method and system. Background Art

[0002] In the medical industry, different hospitals and medical institutions have a large amount of data on patient diagnoses, treatments, etc. This data is not only huge in scale but also contains patients' privacy information. Through federated learning, each medical institution can collaboratively train a more accurate disease classification model without disclosing patients' private data, improving the accuracy and efficiency of medical image data classification. This way of realizing multi-party collaborative training of models without sharing the original data in federated learning not only protects data privacy but also makes full use of the data resources of all parties, enhancing the performance and generalization ability of the model.

[0003] However, with the continuous expansion and deepening of application scenarios, traditional federated learning has gradually exposed limitations when facing the heterogeneity problem of data from different clients. Since the distribution of medical image data from different clients often has significant differences, and the features, labels, and sample numbers of the data are all different, this highly non-independent and identically distributed characteristic makes it difficult for the global model trained by traditional federated learning to achieve ideal performance on each client. The personalized needs of each client cannot be fully met, resulting in a significant reduction in the effectiveness of the disease classification model in actual applications.

[0004] In traditional federated learning and some personalized federated learning, the aggregation method of the disease classification model is relatively simple and often uses average aggregation. In actual scenarios, the data of each client varies greatly in both quantity and feature distribution, and in the existing personalized federated learning during the local aggregation process, a fixed aggregation rule is usually adopted, without fully combining the local data characteristics and actual needs of the client for adaptive adjustment, and the personalized expression ability of the disease classification model is poor. Summary of the Invention

[0005] The present invention provides a clustering-based personalized federated learning image classification method and system, which can solve the non-independent and identically distributed problem in the medical image data of each client and enhance the personalized expression of the local model within each client.

[0006] To achieve the above object, a clustering-based personalized federated learning image classification method provided by the present invention includes:

[0007] The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server, where the local model parameters include deep parameters and shallow parameters;

[0008] The server receives local model parameters and performs clustering operations within a preset period. Among them, a complete clustering operation is performed in the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches the preset number of iterations;

[0009] When the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in the periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain the final clustering parameter cluster;

[0010] Use the federated averaging algorithm to calculate the global model parameters of the final clustering parameter cluster, and calculate the semi-global model parameters of each parameter cluster in the final clustering parameter cluster;

[0011] Calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client;

[0012] The proxy client receives the semi-global model parameters and the global model parameters and performs local adaptive aggregation operations using local medical image data. After the aggregation is completed, a personalized image classification model is obtained, and the personalized image classification model is distributed to the clients other than the proxy client within the parameter cluster where the proxy client is located.

[0013] Optionally, the complete clustering operation performed in the first period within the preset period includes:

[0014] Use a pre-constructed clustering algorithm to divide all clients into K clustering clusters, and randomly select K local model parameter points as the initial clustering centers, where K is a positive integer;

[0015] Calculate the distance from the shallow-layer parameters of each client to the initial clustering center, and allocate the shallow-layer parameters of each client to the initial clustering center closest to the shallow-layer parameters of each client according to the distance to obtain an updated clustering parameter cluster;

[0016] Calculate the mean value of all local model parameter points in the updated clustering parameter cluster to obtain the final clustering center, and divide the complete clustering parameter cluster based on the final clustering center.

[0017] Optionally, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in the periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain an incremental clustering parameter cluster, including:

[0018] When the client updates the shallow-layer parameters in the local model parameters, the client uploads the updated shallow-layer parameters to the server;

[0019] The server receives the updated shallow parameters and uses incremental clustering operations to partition the updated shallow parameters into the complete clustering parameter clusters to obtain the final clustering parameter clusters.

[0020] Optionally, calculating the semi-global model parameters of each parameter cluster in the final clustering parameter clusters includes:

[0021] Calculating the semi-global model parameter Ω of each parameter cluster in the final clustering parameter clusters using the following formula simi-k :

[0022]

[0023] where N i is the parameter sample size of the i-th client, i is the client number, and w i is the local model parameter of the i-th client, and c k is the k-th final clustering parameter cluster.

[0024] Optionally, calculating the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster includes:

[0025] Querying the parameter sample sizes of the clients within each parameter cluster and calculating the ratio of the parameter sample size to the total parameter sample size of all clients to obtain the proportion score;

[0026] Calculating the distance between the local model parameters and the semi-global model parameters of the clients within each parameter cluster to obtain the distance score;

[0027] Calculating the model accuracy of the local models of the clients within each parameter cluster to obtain the accuracy score;

[0028] Performing weighted processing on the proportion score, the distance score, and the accuracy score to obtain the client score of each client.

[0029] Optionally, the proxy client receives the semi-global model parameters and the global model parameters and performs local adaptive aggregation operations using the local medical image data, including:

[0030] Aggregating the deep parameters using the adaptive aggregation formula to obtain the aggregated deep parameters;

[0031] The proxy client collects local medical image data from the local as the local medical subset data, and performs parameter splicing on the aggregated deep parameters and the shallow parameters in the proxy client to obtain the aggregated local model parameters;

[0032] Training the aggregated local model parameters using the local medical subset data, and freezing the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters, and then updating the deep parameter weight matrix in the adaptive aggregation formula;

[0033] Update the aggregated deep parameters in reverse according to the deep network weight matrix. After the reverse update reaches the preset number of iterations, the final aggregated deep parameters are obtained;

[0034] After parameter splicing of the final aggregated deep parameters and the shallow parameters in the proxy client, the final aggregated local model parameters are obtained.

[0035] Optionally, after the aggregation, a personalized image classification model is obtained, including:

[0036] Send the final aggregated local model parameters to the clients within the current parameter cluster. The clients use the final aggregated local model parameters to update the local model parameters and then use the local medical image data collected in the clients for model training. After the training is completed, a personalized image classification model is obtained.

[0037] To solve the above problems, the present invention also provides a device for a clustering-based personalized federated learning image classification method. The device includes: an input module for collecting local medical image data of a client; a classification module deployed with a personalized image classification model obtained by the clustering-based personalized federated learning image classification method, and the personalized image classification model processes the local medical image data to obtain a classification result.

[0038] To solve the above problems, the present invention also provides a system for a clustering-based personalized federated learning image classification method. The system includes a server and one or more clients connected to the server.

[0039] Optionally, the clients store local medical image data and local image classification models. The server performs clustering operations on the model parameters and distributes the model parameters to each client.

[0040] The present invention utilizes a client to train a local image classification model on medical image data, which can ensure the privacy and security of medical image data. Distinguishing deep parameters and shallow parameters can optimize the expression capabilities of different levels of the model in a targeted manner. The server receives the local model parameters and performs clustering within a preset period, which can reflect the data distribution differences of different client groups and provide a basis for personalized models. In addition, when the client updates the local model parameters and uploads them to the server, the server performs an incremental clustering operation, which can avoid global clustering multiple times, reduce the computational load, and improve the computational efficiency. Moreover, calculating the semi-global model parameters for each parameter cluster in the final clustering parameter cluster can ensure that the semi-global model parameters better fit the medical image data distribution of the clients within the cluster. Additionally, selecting the client with the highest score as the proxy client can reduce the number of clients directly participating in the federated aggregation, and the scoring mechanism ensures that the proxy client can effectively represent the group within the cluster. Finally, by having the proxy client receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data, the semi-global model and the global model can be combined to adapt to the local medical image data, solve the non-independent and identically distributed problem in the medical image data of each client, and enhance the personalized expression of the local model within each client. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. is a schematic flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0042] Figure 2 FIG. is a system structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0043] Figure 3 FIG. is an algorithm flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0044] Figure 4 FIG. is a device structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0045] The implementation, functional features, and advantages of the objectives of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] It should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0047] The embodiments of the present application provide a clustering-based personalized federated learning image classification method. The execution subjects of the clustering-based personalized federated learning image classification method include, but are not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiments of the present application. In other words, the clustering-based personalized federated learning image classification method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0048] Referring to Figure 1 As shown, it is a schematic flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention. In this embodiment, the clustering-based personalized federated learning image classification method includes:

[0049] S1. The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server. Among them, the local model parameters include deep-layer parameters and shallow-layer parameters.

[0050] In the embodiments of the present invention, medical image data refers to medical image data collected by the client. Different hospitals and medical institutions have a large amount of data on the diagnosis and treatment of patients. These data are not only huge in scale but also contain patients' privacy information. The medical image data of different hospitals and medical institutions are often non-independent and identically distributed data.

[0051] Exemplarily, the uploaded local model parameters to the server are the trained local model parameters w i =(l i ,h i ), where l i represents shallow-layer parameters, and h i represents deep-layer parameters.

[0052] S2. The server receives the local model parameters and performs clustering operations within a preset period. Among them, a complete clustering operation is performed in the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches the preset number of iterations.

[0053] In the embodiments of the present invention, within the preset period refers to the number of rounds of clustering operations.

[0054] As an embodiment of the present invention, in the first cycle within a preset cycle, a complete clustering operation is performed for complete clustering, including:

[0055] Using a pre-constructed clustering algorithm to divide all clients into K clustering clusters, and randomly selecting K local model parameter points as initial clustering centers, where K is a positive integer;

[0056] Calculate the distance from the shallow parameters of each client to the initial clustering center, and allocate the shallow parameters of each client to the initial clustering center closest to the shallow parameters of each client according to the distance, to obtain an updated clustering parameter cluster;

[0057] Calculate the mean value of all local model parameter points in the updated clustering parameter cluster to obtain the final clustering center, and divide the complete clustering parameter cluster based on the final clustering center.

[0058] In the embodiment of the present invention, the preset cycle refers to taking the number of rounds of multi-round parameter updates as a cycle. For example, the preset cycle is 5 rounds of parameter update rounds.

[0059] In the embodiment of the present invention, the complete clustering operation refers to an operation of clustering all parameters. For example, in the above-mentioned preset cycle of 5 rounds of parameter update rounds, a complete clustering operation is performed in the first round of parameter update rounds.

[0060] Exemplarily, in the first cycle within a preset cycle, to perform a complete clustering operation, the following implementation steps can be adopted:

[0061] S201: Use the K-Means clustering algorithm to divide all clients into K clusters C k , k = 1, 2,... K, and randomly select K model parameter points as the initial clustering centers μ = {μ1, μ2,... μ k}.

[0062] S202: For each shallow parameter l i , calculate the distance between each shallow parameter l i and each clustering center. Let d be the distance metric between the shallow parameters of the client and the cluster center (using the Euclidean distance), and the clustering objective is to minimize where μ k is the center of cluster C k . d(l i , μ k ) = ‖l i - μ k ‖2. Then allocate l i to the cluster corresponding to the clustering center closest to each shallow parameter l i , that is:

[0063] S203: For each cluster, calculate the mean of all data points within the cluster as the new clustering center μ k :

[0064] S204: Continuously repeat the steps of assigning data points and updating the clustering center until the clustering center no longer changes or reaches the preset number of iterations.

[0065] S3. When the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations during cycles other than the first cycle within the preset cycle to update the complete clustering parameter clusters to obtain the final clustering parameter clusters.

[0066] As an embodiment of the present invention, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations during cycles other than the first cycle within the preset cycle to update the complete clustering parameter clusters to obtain incremental clustering parameter clusters, including:

[0067] When the client updates the shallow-layer parameters in the local model parameters, the client uploads the updated shallow-layer parameters to the server;

[0068] The server receives the updated shallow-layer parameters and uses incremental clustering operations to divide the updated shallow-layer parameters into the complete clustering parameter clusters to obtain the final clustering parameter clusters.

[0069] In the embodiments of the present invention, the incremental clustering operation refers to, on the basis of the complete clustering operation, the client clustering the local model parameters updated and uploaded by the client again. For example, when the above preset cycle is 5 rounds of parameter update rounds, after performing the complete clustering operation in the first round of parameter update rounds to obtain the complete clustering parameter clusters, 4 rounds of incremental clustering operations are performed after the first round of parameter update rounds, which can reduce resource consumption.

[0070] Exemplarily, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations during cycles other than the first cycle within the preset cycle to update the complete clustering parameter clusters to obtain incremental clustering parameter clusters, and adopts the following implementation steps:

[0071] S203-new: For the newly uploaded client shallow-layer parameter l i-new , calculate its distance d(l i-new , μ k ) from the existing cluster centers;

[0072] S204-new: Assign l i-new to the cluster with the closest distance and update the center of the cluster

[0073] S4. Use the Federated Averaging algorithm to calculate the global model parameters of the final clustering parameter clusters, and calculate the semi-global model parameters of each parameter cluster in the final clustering parameter clusters.

[0074] In the embodiments of the present invention, the Federated Averaging algorithm means that the client trains the model locally and only uploads the model parameters to the server for weighted average aggregation.

[0075] In the embodiments of the present invention, the semi-global model refers to a parameter optimization method that combines the global model consistency and local data adaptability.

[0076] As an embodiment of the present invention, calculating the semi-global model parameters of each parameter cluster in the final clustering parameter clusters includes:

[0077] Use the following formula to calculate the semi-global model parameter Ω of each parameter cluster in the final clustering parameter clusters simi-k :

[0078]

[0079] where N i is the parameter sample size of the i-th client, i is the client number, w i is the local model parameter of the i-th client, and c k is the k-th final clustering parameter cluster.

[0080] S5. Calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client.

[0081] As an embodiment of the present invention, calculating the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster includes:

[0082] Query the parameter sample sizes of the clients within each parameter cluster, and calculate the ratio of the parameter sample size to the total parameter sample size of all clients to obtain the ratio score;

[0083] Calculate the distance between the local model parameters and the semi-global model parameters of the clients within each parameter cluster to obtain the distance score;

[0084] Calculate the model accuracy of the local models of the clients within each parameter cluster to obtain the accuracy score;

[0085] Perform weighted processing on the ratio score, the distance score, and the accuracy score to obtain the client score of each client.

[0086] Furthermore, perform weighted processing on the ratio score, the distance score, and the accuracy score using the following formula:

[0087] Score i = x1α i + x2β i + x3acc i where acc i is the model accuracy rate, x1 is the first hyperparameter, x2 is the second hyperparameter, x3 is the third hyperparameter, and x1 + x2 + x3 = 1, and Score i is the client score of the i-th client, α i is the proportion score of the i-th client, β i is the distance score of the i-th client.

[0088] Exemplarily, to calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster and select the client with the highest score as the proxy client, the following implementation steps can be adopted:

[0089] S401: Proportion of the sample size within the cluster: Let the parameter sample size of client i be N i , and the total parameter sample size of all clients within cluster C k be N total-k , then

[0090] S402: Distance from the semi-global model parameters: Calculate the distance d(w i between the local model parameters w of client i simi-k and the semi-global model parameters Ω i , Ω simi-k ). To make the score smaller as the distance gets farther, transform the distance

[0091] S403: The comprehensive score is obtained by weighted summation of three factors Score i = x1α i + x2β i + x3acc i where acc i is the model accuracy rate, and x1, x2, x3 are hyperparameters, and x1 + x2 + x3 = 1.

[0092] S404: Select the client with the highest score as the proxy client

[0093] Furthermore, the server sends the semi-global model parameters and the global model parameters to the proxy client, adopting the following steps:

[0094] The server sends the semi-global model parameters Ω simi-k = (l simi-k , h simi-k) and the global model parameter w global =(l, h) is sent to the proxy clients in each cluster, where l simi-k is the semi-global shallow parameter, h simi-k is the semi-global deep parameter, l is the global shallow parameter, and h is the global deep parameter.

[0095] S6. The proxy clients receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data. After the aggregation is completed, a personalized image classification model is obtained, and the personalized image classification model is distributed to the clients in the parameter cluster where the proxy client is located except the proxy client.

[0096] As an embodiment of the present invention, the proxy clients receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data, including:

[0097] Aggregate the deep parameters using the adaptive aggregation formula to obtain the aggregated deep parameters;

[0098] The proxy client collects local medical image data from the local as local medical subset data, and performs parameter splicing on the aggregated deep parameters and the shallow parameters in the proxy client to obtain the aggregated local model parameters;

[0099] Train the aggregated local model parameters using the local medical subset data, and freeze the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters, and then update the deep network weight matrix in the adaptive aggregation formula;

[0100] Backward update the aggregated deep parameters according to the deep network weight matrix, and obtain the final aggregated deep parameters after the backward update reaches the preset number of iterations;

[0101] Perform parameter splicing on the final aggregated deep parameters and the shallow parameters in the proxy client to obtain the final aggregated local model parameters.

[0102] Exemplarily, the proxy clients receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data. The following implementation steps can be adopted:

[0103] S601: Sample the local medical subset data D sub-k .

[0104] S602: Adaptively aggregate the deep parameter h avg =y1h simi-k +(1 - y1)h. y1 is the depth parameter weight matrix that needs to be learned and adjusted, and the scale is the same as h.

[0105] S603: Use the local medical subset data Dsub-k Training the aggregation model W k = [l; h avg , where [;] represents the concatenation of model parameters. Using the backpropagation algorithm and the gradient descent method, the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters are frozen to train and update y1.

[0106] S604: After reaching a certain number of rounds, send the aggregation model parameters W k to other clients within the cluster.

[0107] Furthermore, after the aggregation is completed, a personalized image classification model is obtained, including:

[0108] Send the final aggregated local model parameters to the clients within the current parameter cluster. After the clients update their local model parameters using the final aggregated local model parameters, they use the local medical image data collected within the clients for model training. After the training is completed, a personalized image classification model is obtained.

[0109] Furthermore, after receiving the personalized image classification model, the clients other than the proxy client use their own local medical image data to continue the model parameter optimization process of steps S1 - S6 for the personalized image classification model until the number of times of model parameter optimization in the personalized image classification models of the clients within the parameter cluster where the proxy client is located reaches the preset maximum number of optimization times, and then the final personalized image classification model is obtained.

[0110] In the embodiments of the present invention, the preset maximum number of optimization times refers to the number of optimization times that enables the model performance of the personalized image classification model to converge.

[0111] The present invention utilizes a client to train a local image classification model on medical imaging data, which can ensure the privacy and security of medical imaging data. Distinguishing deep parameters and shallow parameters can optimize the expression capabilities of different levels of the model. The server receives local model parameters and performs clustering within a preset period, which can reflect the differences in data distributions among different client groups and provide a basis for personalized models. In addition, when the client updates local model parameters and uploads them to the server, the server performs incremental clustering operations, which can avoid global clustering multiple times, reduce the computational load, and improve the computational efficiency. Moreover, using the federated averaging algorithm to calculate the semi-global model parameters of each parameter cluster in the final clustering parameter cluster can ensure that the semi-global model parameters better fit the distribution of medical imaging data of clients within the cluster. Additionally, selecting the client with the highest score as the proxy client can reduce the number of clients directly participating in federated aggregation, and the scoring mechanism ensures that the proxy client can effectively represent the group within the cluster. Finally, by having the proxy client receive the semi-global model parameters and global model parameters and perform local adaptive aggregation operations using local medical imaging data, it is possible to combine the semi-global model and the global model, adapt to local medical imaging data, solve the problem of non-independent and identically distributed medical imaging data among clients, and enhance the personalized expression of local models within each client.

[0112] Referring to Figure 2 As shown, it is the system structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0113] The system includes a server and one or more clients connected to the server.

[0114] Furthermore, the client includes local medical imaging data stored in the client and a local model stored in the client.

[0115] Furthermore, the server performs clustering operations on the model parameters and distributes the model parameters to each client.

[0116] Referring to Figure 3 As shown, it is the algorithm flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0117] Referring to Figure 4 As shown, it is the device structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0118] As an embodiment of the present invention, the device includes: an input module for collecting local medical imaging data of the client; a classification module deployed with a personalized image classification model obtained by a clustering-based personalized federated learning image classification method, and the personalized image classification model processes the local medical imaging data to obtain a classification result.

[0119] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0120] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any associated drawing reference signs in the claims should not be construed as limiting the claimed rights.

[0121] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain, in essence, is a decentralized database, a series of data blocks generated by using cryptographic methods. Each data block contains information on a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, an application service layer, etc.

[0122] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0123] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. Words such as first, second, etc. are used to denote names and do not represent any specific order.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A clustering-based personalized federated learning image classification method, characterized in that The method includes: The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server, where the local model parameters include deep parameters and shallow parameters; The server receives the local model parameters and performs a clustering operation within a preset period. Specifically, a complete clustering operation is performed in the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches the preset number of iterations; When the client updates the local model parameters and uploads them to the server, the server performs an incremental clustering operation in periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain a final clustering parameter cluster; Use the federated averaging algorithm to calculate the global model parameters of the final clustering parameter cluster, and calculate the semi-global model parameters of each parameter cluster in the final clustering parameter cluster; Calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client; The proxy client receives the semi-global model parameters and the global model parameters and performs a local adaptive aggregation operation using the local medical image data. After the aggregation is completed, a personalized image classification model is obtained, and the personalized image classification model is distributed to the clients other than the proxy client within the parameter cluster where the proxy client is located.

2. The personalized federated learning image classification method based on clustering according to claim 1, wherein The complete clustering operation performed in the first period within the preset period includes: Using a pre-constructed clustering algorithm to divide all clients into K clustering clusters, and randomly selecting K local model parameter points as the initial clustering centers, where K is a positive integer; Calculate the distance from the shallow parameters of each client to the initial clustering centers, and assign the shallow parameters of each client to the initial clustering center closest to the shallow parameters of each client according to the distance to obtain an updated clustering parameter cluster; Calculate the mean value of all local model parameter points in the updated clustering parameter cluster to obtain the final clustering centers, and divide the complete clustering parameter cluster based on the final clustering centers.

3. The clustering-based personalized federated learning image classification method according to claim 1 or 2, characterized in that, When the client updates the local model parameters and uploads them to the server, the server performs an incremental clustering operation in periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain an incremental clustering parameter cluster, including: When the client updates the shallow parameters in the local model parameters, the client uploads the updated shallow parameters to the server; The server receives the updated shallow parameters and uses the incremental clustering operation to divide the updated shallow parameters into the complete clustering parameter cluster to obtain the final clustering parameter cluster.

4. The personalized federated learning image classification method based on clustering according to claim 1, wherein The calculation of the semi-global model parameters of each parameter cluster in the final clustering parameter cluster includes: Use the following formula to calculate the semi-global model parameter Ω of each parameter cluster in the final clustering parameter cluster simi-k : Among them, i is the client number in the k-th final clustering parameter cluster, N i is the parameter sample size of the i-th client, w i is the local model parameter of the i-th client, c k is the k-th final clustering parameter cluster, and i, k are positive integers.

5. The personalized federated learning image classification method based on clustering according to claim 1, wherein, The calculation of the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster includes: Query the parameter sample size of the clients within each parameter cluster, and calculate the ratio of the parameter sample size to the total parameter sample size of all clients to obtain a ratio score; Calculate the distance between the local model parameters and the semi-global model parameters of the clients within each parameter cluster to obtain a distance score; Calculate the model accuracy of the local models of the clients within each parameter cluster to obtain an accuracy score. Perform weighted processing on the proportion score, distance score, and accuracy score to obtain a client score for each client.

6. The personalized federated learning image classification method based on clustering according to claim 1, characterized in that, The proxy client receives semi-global model parameters and global model parameters and performs local adaptive aggregation operations using local medical image data, including: Aggregate deep parameters using the adaptive aggregation formula to obtain aggregated deep parameters. The proxy client collects local medical image data from the local area as local medical subset data, and performs parameter splicing on the aggregated deep parameters and the shallow parameters in the proxy client to obtain aggregated local model parameters. Train the aggregated local model parameters using the local medical subset data, freeze the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters, and then update the deep parameter weight matrix in the adaptive aggregation formula. Backward update the aggregated deep parameters according to the deep network weight matrix, and obtain the final aggregated deep parameters after the backward update reaches the preset number of iterations. Perform parameter splicing on the final aggregated deep parameters and the shallow parameters in the proxy client to obtain the final aggregated local model parameters.

7. The personalized federated learning image classification method based on clustering according to claim 1 or 6, characterized in that After the aggregation is completed, a personalized image classification model is obtained, including: Send the final aggregated local model parameters to the clients within the current parameter cluster. The clients update their local model parameters using the final aggregated local model parameters and then perform model training using the local medical image data collected within the clients. After the training is completed, a personalized image classification model is obtained.

8. A personalized federated learning image classification device based on clustering, characterized in that, Including: An input module for collecting local medical image data of the client. A classification module deployed with a personalized image classification model obtained by the method according to any one of claims 1-7. The personalized image classification model processes the local medical image data to obtain a classification result.

9. A system for implementing the clustering-based personalized federated learning image classification method according to any one of claims 1 to 7. The system includes a server and one or more clients connected to the server.

10. The system for the clustering-based personalized federated learning image classification method according to claim 9. The client stores local medical image data and a local image classification model. The server performs clustering operations on the model parameters and distributes the model parameters to each client.

Images