Clustering-based personalized federal learning image classification method and system

Abstract

The invention relates to an artificial intelligence technology, and discloses a clustering-based personalized federal learning image classification method and system, and the method comprises the steps: a client generates local model parameters, and uploads the local model parameters to a server; the server receives the local model parameters and performs complete clustering, and when the client updates the local model parameters, the server executes incremental clustering operation to obtain a final clustering parameter cluster; calculating a semi-global model parameter of each parameter cluster and calculating a client score to select a proxy client, and sending the semi-global model parameter and the global model parameter to the proxy client by the server; and the proxy client carries out local adaptive aggregation operation, and after aggregation is completed, a personalized image classification model is obtained. The invention further provides a device and a system of the clustering-based personalized federal learning image classification method. According to the method, the problem of non-independent identical distribution in the medical image data of each client can be solved, and the personalized expression of the local model in each client is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and particularly relates to a clustering-based personalized federated learning image classification method and system. Background Art

[0002] In the medical industry, different hospitals and medical institutions have a large amount of data on patient diagnosis, treatment, etc. This data is not only huge in scale but also contains patients' privacy information. Through federated learning, each medical institution can collaboratively train a more accurate disease classification model without disclosing patients' private data, improving the accuracy and efficiency of medical image data classification. This way of realizing multi-party collaborative model training without sharing the original data in federated learning not only protects data privacy but also makes full use of the data resources of all parties, enhancing the performance and generalization ability of the model.

[0003] However, with the continuous expansion and deepening of application scenarios, traditional federated learning has gradually revealed limitations when facing the heterogeneity problem of data from different clients. Since the distribution of medical image data from different clients often has significant differences, the features, labels, and sample numbers of the data are not the same. This highly non-independent and identically distributed characteristic makes it difficult for the global model trained by traditional federated learning to achieve ideal performance on each client. The personalized needs of each client cannot be fully met, resulting in a significant reduction in the effect of the disease classification model in actual applications.

[0004] In traditional federated learning and some personalized federated learning, the aggregation method of the disease classification model is relatively simple and often uses average aggregation. In actual scenarios, the data of each client varies greatly in both quantity and feature distribution. And in the existing personalized federated learning during the local aggregation process, a fixed aggregation rule is usually adopted, without fully combining the local data characteristics and actual needs of the client for adaptive adjustment, so the personalized expression ability of the disease classification model is poor. Summary of the Invention

[0005] The present invention provides a clustering-based personalized federated learning image classification method and system, which can solve the non-independent and identically distributed problem in the medical image data of each client and enhance the personalized expression of the local model within each client.

[0006] To achieve the above object, a clustering-based personalized federated learning image classification method provided by the present invention includes:

[0007] The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server, where the local model parameters include deep parameters and shallow parameters;

[0008] The server receives local model parameters and performs clustering operations within a preset period. Among them, a complete clustering operation is performed in the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches the preset number of iterations;

[0009] When the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in the periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain the final clustering parameter cluster;

[0010] Use the federated averaging algorithm to calculate the global model parameters of the final clustering parameter cluster, and calculate the semi-global model parameters of each parameter cluster in the final clustering parameter cluster;

[0011] Calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client;

[0012] The proxy client receives the semi-global model parameters and the global model parameters and performs local adaptive aggregation operations using the local medical image data. After the aggregation is completed, a personalized image classification model is obtained, and the personalized image classification model is distributed to the clients other than the proxy client within the parameter cluster where the proxy client is located.

[0013] Optionally, the complete clustering operation performed in the first period within the preset period includes:

[0014] Use a pre-constructed clustering algorithm to divide all clients into K clustering clusters, and randomly select K local model parameter points as the initial clustering centers, where K is a positive integer;

[0015] Calculate the distance from the shallow-layer parameters of each client to the initial clustering centers, and assign the shallow-layer parameters of each client to the initial clustering center closest to the shallow-layer parameters of each client according to the distance to obtain an updated clustering parameter cluster;

[0016] Calculate the mean value of all local model parameter points in the updated clustering parameter cluster to obtain the final clustering center, and divide the complete clustering parameter cluster based on the final clustering center.

[0017] Optionally, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in the periods other than the first period within the preset period to update the complete clustering parameter cluster to obtain an incremental clustering parameter cluster, including:

[0018] When the client updates the shallow-layer parameters in the local model parameters, the client uploads the updated shallow-layer parameters to the server;

[0019] The server receives the updated shallow parameters and uses incremental clustering operations to partition the updated shallow parameters into the complete clustering parameter clusters, obtaining the final clustering parameter clusters.

[0020] Optionally, calculating the semi-global model parameters of each parameter cluster in the final clustering parameter clusters includes:

[0021] Calculating the semi-global model parameter Ω of each parameter cluster in the final clustering parameter clusters using the following formula simi-k :

[0022]

[0023] where N i is the parameter sample size of the i-th client, i is the client number, w i is the local model parameter of the i-th client, c k is the k-th final clustering parameter cluster.

[0024] Optionally, calculating the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster includes:

[0025] Querying the parameter sample sizes of the clients within each parameter cluster and calculating the ratio of the parameter sample size to the total parameter sample size of all clients to obtain the ratio score;

[0026] Calculating the distance between the local model parameters and the semi-global model parameters of the clients within each parameter cluster to obtain the distance score;

[0027] Calculating the model accuracy of the local models of the clients within each parameter cluster to obtain the accuracy score;

[0028] Performing weighted processing on the ratio score, the distance score, and the accuracy score to obtain the client score of each client.

[0029] Optionally, the proxy client receives the semi-global model parameters and the global model parameters and performs local adaptive aggregation operations using the local medical image data, including:

[0030] Aggregating the deep parameters using the adaptive aggregation formula to obtain the aggregated deep parameters;

[0031] The proxy client collects local medical image data from the local as the local medical subset data, and after parameter splicing of the aggregated deep parameters and the shallow parameters in the proxy client, obtains the aggregated local model parameters;

[0032] Training the aggregated local model parameters using the local medical subset data, and freezing the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters, and then updating the deep parameter weight matrix in the adaptive aggregation formula;

[0033] Backward update the aggregated deep parameters according to the deep network weight matrix, and obtain the final aggregated deep parameters after the backward update reaches the preset number of iterations;

[0034] After parameter splicing of the final aggregated deep parameters and the shallow parameters in the proxy client, the final aggregated local model parameters are obtained.

[0035] Optionally, after the aggregation is completed, a personalized image classification model is obtained, including:

[0036] Send the final aggregated local model parameters to the clients within the current parameter cluster. After the clients update the local model parameters using the final aggregated local model parameters, they use the local medical image data collected in the clients for model training, and a personalized image classification model is obtained after the training is completed.

[0037] To solve the above problems, the present invention also provides a device for a clustering-based personalized federated learning image classification method. The device includes: an input module for collecting local medical image data of clients; a classification module deployed with a personalized image classification model obtained by the clustering-based personalized federated learning image classification method, and the personalized image classification model processes the local medical image data to obtain a classification result.

[0038] To solve the above problems, the present invention also provides a system for a clustering-based personalized federated learning image classification method. The system includes a server and one or more clients connected to the server.

[0039] Optionally, the clients store local medical image data and local image classification models. The server performs clustering operations on the model parameters and distributes the model parameters to each client.

[0040] The present invention uses a client to train a local image classification model on medical image data, which can ensure the privacy and security of medical image data. Distinguishing deep parameters and shallow parameters can optimize the expression ability of different levels of the model in a targeted manner. The server receives the local model parameters and performs clustering within a preset period, which can reflect the data distribution differences of different client groups and provide a basis for personalized models. In addition, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations, which can avoid global clustering multiple times, reduce the computational load, and improve the computational efficiency. Moreover, calculating the semi-global model parameters of each parameter cluster in the final clustering parameter cluster can ensure that the semi-global model parameters are more in line with the medical image data distribution of the clients within the cluster. Additionally, selecting the client with the highest score as the proxy client can reduce the number of clients directly participating in the federated aggregation, and the scoring mechanism ensures that the proxy client can effectively represent the group within the cluster. Finally, by having the proxy client receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data, the semi-global model and the global model can be combined to adapt to the local medical image data, solve the non-independent and identically distributed problem in the medical image data of each client, and enhance the personalized expression of the local model within each client. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 FIG. is a schematic flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0042] Figure 2 FIG. is a system structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0043] Figure 3 FIG. is an algorithm flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention;

[0044] Figure 4 FIG. is a device structure diagram of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0045] The realization, functional features, and advantages of the objectives of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] The embodiment of the present application provides a clustering-based personalized federated learning image classification method. The execution subject of the clustering-based personalized federated learning image classification method includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the clustering-based personalized federated learning image classification method can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0048] Referring to Figure 1 As shown, it is a schematic flowchart of a clustering-based personalized federated learning image classification method provided by an embodiment of the present invention. In this embodiment, the clustering-based personalized federated learning image classification method includes:

[0049] S1. The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server. Among them, the local model parameters include deep-layer parameters and shallow-layer parameters.

[0050] In the embodiment of the present invention, medical image data refers to medical image data collected by the client. Different hospitals and medical institutions have a large amount of data on patient diagnosis, treatment, etc. These data are not only huge in scale but also contain patients' privacy information. The medical image data of different hospitals and medical institutions are often non-independent and identically distributed data.

[0051] Exemplarily, the uploaded local model parameters to the server are the trained local model parameters w i =(l i ,h i ), where l i represents the shallow-layer parameters, and h i represents the deep-layer parameters.

[0052] S2. The server receives the local model parameters and performs clustering operations within a preset period. Among them, a complete clustering operation is performed in the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches the preset number of iterations.

[0053] In the embodiment of the present invention, within the preset period refers to the number of rounds of clustering operations.

[0054] As an embodiment of the present invention, a complete clustering operation is performed in the first cycle within a preset cycle, including:

[0055] Using a pre-constructed clustering algorithm to divide all clients into K clustering clusters, and randomly selecting K local model parameter points as initial clustering centers, where K is a positive integer;

[0056] Calculate the distance from the shallow parameters of each client to the initial clustering center, and allocate the shallow parameters of each client to the initial clustering center closest to the shallow parameters of each client according to the distance, to obtain an updated clustering parameter cluster;

[0057] Calculate the mean value of all local model parameter points in the updated clustering parameter cluster to obtain the final clustering center, and divide the complete clustering parameter cluster based on the final clustering center.

[0058] In the embodiment of the present invention, the preset cycle refers to taking the number of rounds of multi-round parameter updates as a cycle. For example, the preset cycle is 5 rounds of parameter update rounds.

[0059] In the embodiment of the present invention, the complete clustering operation refers to an operation of clustering all parameters. For example, in the above-mentioned preset cycle of 5 rounds of parameter update rounds, a complete clustering operation is performed in the 1st round of parameter update rounds.

[0060] Exemplarily, to perform a complete clustering operation in the first cycle within a preset cycle, the following implementation steps can be adopted:

[0061] S201: Use the K-Means clustering algorithm to divide all clients into K clusters C k , k = 1, 2, … K, and randomly select K model parameter points as the initial clustering centers μ = {μ 1 , μ 2 , … μ k}.

[0062] S202: For each shallow parameter l i , calculate the distance between each shallow parameter l i and each clustering center. Let d be the distance metric between the shallow parameters of the client and the cluster center (using the Euclidean distance), and the clustering objective is to minimize where μ k is the center of cluster C k . d(l i , μ k ) = ‖l i - μ k ‖ 2 . Then allocate l i to the initial clustering center closest to each shallow parameter l iIn the cluster corresponding to the nearest cluster center, that is:

[0063] S203: For each cluster, calculate the mean of all data points within the cluster as the new cluster center μ k :

[0064] S204: Continuously repeat the steps of assigning data points and updating the cluster center until the cluster center no longer changes or reaches the preset number of iterations.

[0065] S3. When the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in cycles other than the first cycle within the preset cycle to update the complete clustering parameter cluster to obtain the final clustering parameter cluster.

[0066] As an embodiment of the present invention, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in cycles other than the first cycle within the preset cycle to update the complete clustering parameter cluster to obtain the incremental clustering parameter cluster, including:

[0067] When the client updates the shallow-layer parameters in the local model parameters, the client uploads the updated shallow-layer parameters to the server;

[0068] The server receives the updated shallow-layer parameters and uses incremental clustering operations to divide the updated shallow-layer parameters into the complete clustering parameter cluster to obtain the final clustering parameter cluster.

[0069] In the embodiment of the present invention, the incremental clustering operation refers to, on the basis of the complete clustering operation, the client re-clustering the local model parameters updated and uploaded by the client. For example, when the preset cycle is 5 rounds of parameter update rounds, after performing the complete clustering operation in the first round of parameter update rounds to obtain the complete clustering parameter cluster, 4 rounds of incremental clustering operations are performed after the first round of parameter update rounds, which can reduce resource consumption.

[0070] Exemplarily, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations in cycles other than the first cycle within the preset cycle to update the complete clustering parameter cluster to obtain the incremental clustering parameter cluster, and adopts the following implementation steps:

[0071] S203-new: For the newly uploaded client shallow-layer parameter l i-new , calculate its distance d(l i-new , μ k ) from the existing cluster centers;

[0072] S204-new: Assign l i-newAllocate it to the nearest cluster and update the center of that cluster

[0073] S4. Use the Federated Averaging algorithm to calculate the global model parameters of the final clustering parameter clusters, and calculate the semi-global model parameters of each parameter cluster in the final clustering parameter clusters.

[0074] In the embodiment of the present invention, the Federated Averaging algorithm means that the client trains the model locally and only uploads the model parameters to the server for weighted average aggregation.

[0075] In the embodiment of the present invention, the semi-global model refers to a parameter optimization method that combines the global model consistency and local data adaptability.

[0076] As an embodiment of the present invention, calculating the semi-global model parameters of each parameter cluster in the final clustering parameter clusters includes:

[0077] Use the following formula to calculate the semi-global model parameter Ω of each parameter cluster in the final clustering parameter clusters simi-k :

[0078]

[0079] where N i is the parameter sample size of the i-th client, i is the client number, w i is the local model parameter of the i-th client, and c k is the k-th final clustering parameter cluster.

[0080] S5. Calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client.

[0081] As an embodiment of the present invention, calculating the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster includes:

[0082] Query the parameter sample sizes of the clients within each parameter cluster, and calculate the ratio of the parameter sample size to the total parameter sample size of all clients to obtain the ratio score;

[0083] Calculate the distance between the local model parameters and the semi-global model parameters of the clients within each parameter cluster to obtain the distance score;

[0084] Calculate the model accuracy of the local models of the clients within each parameter cluster to obtain the accuracy score;

[0085] Perform weighted processing on the ratio score, the distance score, and the accuracy score to obtain the client score of each client.

[0086] Furthermore, perform a weighted process on the proportion score, distance score, and accuracy score, using the following formula:

[0087] Score i = x 1 α i + x 2 β i + x 3 acc i , where acc i is the model accuracy, x 1 is the first hyperparameter, x 2 is the second hyperparameter, x 3 is the third hyperparameter, and x 1 + x 2 + x 3 = 1, Score i is the client score of the i-th client, α i is the proportion score of the i-th client, β i is the distance score of the i-th client.

[0088] Exemplarily, calculate the scores of the clients within each parameter cluster according to the semi-global model parameters of each parameter cluster, and select the client with the highest score as the proxy client. The following implementation steps can be adopted:

[0089] S401: Proportion of the sample size within the cluster: Let the parameter sample size of client i be N i , and the total parameter sample size of all clients within cluster C k be N total-k . Then

[0090] S402: Distance from the semi-global model parameters: Calculate the distance d(w i between the local model parameters w of client i and the semi-global model parameters Ω simi-k . To make the score smaller as the distance gets farther, transform the distance i , Ω simi-k ).

[0091] S403: The comprehensive score is obtained by weighted summation of three factors Score i = x 1 α i + x 2 β i + x 3 acc i , where acc i is the model accuracy, x 1 、x 2, x 3 is a hyperparameter, and x 1 + x 2 + x 3 = 1.

[0092] S404: Select the client with the highest score as the proxy client

[0093] Furthermore, the server sends the semi-global model parameters and the global model parameters to the proxy client, using the following steps:

[0094] The server sends the semi-global model parameter Ω simi-k = (l simi-k , h simi-k ) and the global model parameter w global = (l, h) to the proxy client of each cluster, where l simi-k is the semi-global shallow parameter, h simi-k is the semi-global deep parameter, l is the global shallow parameter, and h is the global deep parameter.

[0095] S6. The proxy client receives the semi-global model parameters and the global model parameters and performs a local adaptive aggregation operation using the local medical image data. After the aggregation is completed, a personalized image classification model is obtained, and the personalized image classification model is distributed to the clients in the parameter cluster where the proxy client is located except the proxy client.

[0096] As an embodiment of the present invention, the proxy client receives the semi-global model parameters and the global model parameters and performs a local adaptive aggregation operation using the local medical image data, including:

[0097] Aggregate the deep parameters using the adaptive aggregation formula to obtain the aggregated deep parameters;

[0098] The proxy client collects the local medical image data from the local as the local medical subset data, and performs parameter splicing on the aggregated deep parameters and the shallow parameters in the proxy client to obtain the aggregated local model parameters;

[0099] Train the aggregated local model parameters using the local medical subset data, and freeze the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters, and then update the deep network weight matrix in the adaptive aggregation formula;

[0100] Backward update the aggregated deep parameters according to the deep network weight matrix, and obtain the final aggregated deep parameters after the backward update reaches the preset number of iterations;

[0101] Perform parameter splicing on the final aggregated deep parameters and the shallow parameters in the proxy client to obtain the final aggregated local model parameters.

[0102] Exemplarily, the proxy client receives semi-global model parameters and global model parameters and performs local adaptive aggregation operations using local medical image data. The following implementation steps can be adopted:

[0103] S601: Sample the local medical subset data D sub-k .

[0104] S602: Adaptively aggregate the deep parameter h avg = y 1 h simi-k +(1 - y 1 )h. y 1 y is the weight matrix of the deep parameter that needs to be learned and adjusted, with the same scale as h.

[0105] S603: Use the local medical subset data D sub-k to train the aggregation model W k = [l; h avg , where [;] represents the concatenation of model parameters. The backpropagation algorithm and the gradient descent method are used to freeze the semi-global shallow parameters and semi-global deep parameters in the semi-global model parameters and the deep parameters in the global model parameters to train and update y 1 .

[0106] S604: After reaching a certain number of rounds, send the aggregation model parameters W k to other clients within the cluster.

[0107] Furthermore, after the aggregation is completed, a personalized image classification model is obtained, including:

[0108] Send the final aggregated local model parameters to the clients within the current parameter cluster. The clients use the final aggregated local model parameters to update their local model parameters and then use the local medical image data collected within the clients for model training. After the training is completed, a personalized image classification model is obtained.

[0109] Furthermore, after the clients other than the proxy client receive the personalized image classification model, the clients other than the proxy client use their own local medical image data to continue to perform the model parameter optimization process of steps S1 - S6 on the personalized image classification model until the number of model parameter optimizations in the personalized image classification models of the clients within the parameter cluster where the proxy client is located reaches the preset maximum number of optimizations, and then the final personalized image classification model is obtained.

[0110] In the embodiments of the present invention, the preset maximum number of optimizations refers to the number of optimizations that enables the model performance of the personalized image classification model to converge.

[0111] The present invention utilizes the client to train a local image classification model on medical image data, which can ensure the privacy and security of medical image data. Distinguishing deep parameters and shallow parameters can targetedly optimize the expression capabilities of different levels of the model. The server receives the local model parameters and performs clustering within a preset period, which can reflect the data distribution differences of different client groups and provide a basis for personalized models. In addition, when the client updates the local model parameters and uploads them to the server, the server performs incremental clustering operations, which can avoid global clustering multiple times, reduce the computational load, and improve the computational efficiency. Moreover, using the federated averaging algorithm to calculate the semi-global model parameters of each parameter cluster in the final clustering parameter cluster can ensure that the semi-global model parameters are more in line with the medical image data distribution of the clients within the cluster. Additionally, selecting the client with the highest score as the proxy client can reduce the number of clients directly participating in the federated aggregation, and the scoring mechanism ensures that the proxy client can effectively represent the group within the cluster. Finally, by having the proxy client receive the semi-global model parameters and the global model parameters and perform local adaptive aggregation operations using the local medical image data, the semi-global model and the global model can be combined to adapt to the local medical image data, solve the non-independent and identically distributed problems in the medical image data of each client, and enhance the personalized expression of the local model within each client.

[0112] Referring to Figure 2 As shown, it is the system structure diagram of the clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0113] The system includes a server and one or more clients connected to the server.

[0114] Further, the client includes local medical image data stored in the client and a local model stored in the client.

[0115] Further, the server performs clustering operations on the model parameters and distributes the model parameters to each client.

[0116] Referring to Figure 3 As shown, it is the algorithm flowchart of the clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0117] Referring to Figure 4 As shown, it is the device structure diagram of the clustering-based personalized federated learning image classification method provided by an embodiment of the present invention.

[0118] As an embodiment of the present invention, the device includes: an input module for collecting local medical image data of the client; a classification module deployed with a personalized image classification model obtained by the clustering-based personalized federated learning image classification method, and the personalized image classification model processes the local medical image data to obtain a classification result.

[0119] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention.

[0120] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any associated drawing reference signs in the claims should not be construed as limiting the claims involved.

[0121] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithms. Blockchain, in essence, is a decentralized database, a string of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-counterfeiting) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, an application service layer, etc.

[0122] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use the knowledge to obtain the best results.

[0123] In addition, obviously, the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. Words such as first, second, etc. are used to denote names and do not denote any specific order.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A clustering-based personalized federated learning image classification method, characterized in that: The method comprises: The client uses local medical image data to train a local image classification model to obtain local model parameters, and uploads the local model parameters to the server, where the local model parameters include deep parameters and shallow parameters; The server receives the local model parameters and performs a clustering operation within a preset period, wherein a complete clustering operation is performed within the first period within the preset period, and a complete clustering parameter cluster is obtained when the clustering reaches a preset number of iterations; When the client updates the local model parameters and uploads them to the server, the server performs an incremental clustering operation in a cycle other than the first cycle in a preset cycle to update the complete clustering parameter cluster to obtain a final clustering parameter cluster; The global model parameters of the final clustering parameter cluster are calculated using the federated average algorithm, and the semi-global model parameters of each parameter cluster in the final clustering parameter cluster are calculated; The score of the client in each parameter cluster is calculated according to the semi-global model parameters of each parameter cluster, and the client with the highest score is selected as the proxy client. The server sends the semi-global model parameters and the global model parameters to the proxy client. The proxy client receives the semi-global model parameters and the global model parameters and uses the local medical image data to perform local adaptive aggregation operations. After the aggregation is completed, a personalized image classification model is obtained and the personalized image classification model is distributed to the clients other than the proxy client in the parameter cluster where the proxy client is located.

2. The clustering-based personalized federated learning image classification method according to claim 1, characterized in that: The complete clustering operation is performed in the first cycle within the preset cycle, including: Use the pre-built clustering algorithm to divide all clients into K clusters, and randomly select K local model parameter points as the initial cluster centers, where K is a positive integer; Calculate the distance between the shallow parameters of each client and the initial clustering center, and assign the shallow parameters of each client to the initial clustering center closest to the shallow parameters of each client according to the distance to obtain an updated clustering parameter cluster; The average value of all local model parameter points in the updated clustering parameter cluster is calculated to obtain the final cluster center, and the complete clustering parameter cluster is divided based on the final cluster center.

3. The clustering-based personalized federated learning image classification method according to claim 1 or 2, characterized in that: When the client updates the local model parameters and uploads them to the server, the server performs an incremental clustering operation in a period other than the first period in a preset period to update the complete clustering parameter cluster to obtain an incremental clustering parameter cluster, including: When the client updates the shallow parameters in the local model parameters, the client uploads the updated shallow parameters to the server; The server receives the updated shallow parameters and uses an incremental clustering operation to divide the updated shallow parameters into complete clustering parameter clusters to obtain the final clustering parameter clusters.

4. The clustering-based personalized federated learning image classification method according to claim 1, characterized in that: The calculating of the semi-global model parameters of each parameter cluster in the final clustering parameter cluster comprises: The semi-global model parameter Ω of each parameter cluster in the final clustering parameter cluster is calculated using the following formula: simi-k : Where i is the client number in the kth final clustering parameter cluster, N i is the parameter sample size of the i-th client, w i is the local model parameter of the ith client, c k is the kth final clustering parameter cluster, i, k are positive integers.

5. The clustering-based personalized federated learning image classification method according to claim 1, characterized in that: The step of calculating the score of the client in each parameter cluster according to the semi-global model parameters of each parameter cluster includes: Query the parameter sample size in the client of each parameter cluster, and calculate the ratio of the parameter sample size to the total parameter sample size in all clients to obtain the percentage score; Calculate the distance between the local model parameters of the client and the semi-global model parameters in each parameter cluster to obtain the distance score; Calculate the model accuracy of the local model of the client in each parameter cluster to obtain the accuracy score; The proportion score, distance score, and accuracy score are weighted to obtain the client score of each client.

6. The clustering-based personalized federated learning image classification method according to claim 1, characterized in that: The proxy client receives the semi-global model parameters and the global model parameters and uses the local medical image data to perform a local adaptive aggregation operation, including: Aggregating deep parameters using an adaptive aggregation formula to obtain aggregated deep parameters; The proxy client collects medical imaging data from the local as local medical subset data, and concatenates the aggregated deep parameters and the shallow parameters in the proxy client to obtain the aggregated local model parameters; The local medical subset data is used to train the aggregated local model parameters, and the semi-global shallow parameters and the semi-global deep parameters in the semi-global model parameters are frozen, and the deep parameters in the global model parameters are frozen, and then the deep parameter weight matrix in the adaptive aggregation formula is updated; Reversely update the aggregated deep parameters according to the deep network weight matrix, and obtain the final aggregated deep parameters after the reverse update reaches a preset number of iterations; The final aggregated deep parameters and the shallow parameters in the proxy client are concatenated to obtain the final aggregated local model parameters.

7. The clustering-based personalized federated learning image classification method according to claim 1 or 6, characterized in that: After the aggregation is completed, a personalized image classification model is obtained, including: The final aggregated local model parameters are sent to the client in the current parameter cluster. The client updates the local model parameters using the final aggregated local model parameters and then uses the local medical imaging data collected in the client to perform model training. After the training is completed, a personalized image classification model is obtained.

8. A clustering-based personalized federated learning image classification device, characterized in that: include: An input module, used to collect local medical imaging data from the client; The classification module is deployed with a personalized image classification model obtained according to the method described in any one of claims 1 to 7, and the personalized image classification model processes local medical image data to obtain a classification result.

9. A system for implementing the clustering-based personalized federated learning image classification method as described in any one of claims 1 to 7, wherein the system comprises a server and one or more clients connected to the server.

10. A system for a clustering-based personalized federated learning image classification method as described in claim 9, wherein the client stores local medical imaging data and a local image classification model, and the server performs clustering operations on model parameters and distributes the model parameters to each client.

Images