Federal learning data processing method and device and storage medium

Abstract

The embodiment of the invention provides a federal learning data processing method and device, a storage medium and a computer program product, and belongs to the technical field of machine learning. Comprising the following steps: distributing an initial global model to a plurality of clients; receiving model gradient information uploaded by each client after training the initial global model based on the local data set; multiple candidate clustering schemes are randomly generated, and each candidate clustering scheme corresponds to a grouping mode for dividing all clients into different clusters; determining a client set in each cluster according to the candidate clustering scheme, calculating an aggregation gradient of each cluster, and determining an evaluation index of the candidate clustering scheme based on the aggregation gradients of all clusters and the independent data set for model evaluation; determining a target user clustering scheme from a plurality of candidate clustering schemes according to the evaluation index; and grouping the clients based on the target user clustering scheme, and performing model aggregation training on each group to obtain a final model corresponding to each group.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, specifically to a federated learning data processing method, apparatus, storage medium, and computer program product. Background Technology

[0002] Federated learning, as a privacy-preserving distributed machine learning paradigm, enables collaborative modeling across multiple clients while ensuring local data storage, and has demonstrated significant value in fields such as healthcare and finance. However, in real-world scenarios, client data often exhibits non-independent and identically distributed characteristics, meaning it shows significant heterogeneity in sample size, category distribution, and feature dimensions, leading to conflicts between the optimization objectives of local and global models. A single global model struggles to simultaneously adapt to various differentiated needs, severely limiting the generalization performance and application effectiveness of federated learning in complex practical tasks.

[0003] Existing research primarily addresses the challenge of data heterogeneity by grouping clients into clusters and training personalized models for each group. However, these methods often rely on pre-defined cluster numbers or fixed distribution assumptions, making them ill-suited to dynamically changing client data characteristics. Furthermore, they frequently require exchanging complete model parameters or rely on highly complex data similarity calculations, resulting in significant communication overhead and slow convergence. Simultaneously, encryption or perturbation mechanisms introduced to enhance privacy often further sacrifice model accuracy or increase computational burden, making it difficult to achieve an effective balance between performance, efficiency, and security. Summary of the Invention

[0004] The purpose of this application is to provide a federated learning data processing method, apparatus, and storage medium. It aims to address the technical shortcomings of existing federated clustering methods in scenarios with non-independent and identically distributed data. These shortcomings include: difficulty in adaptively determining the optimal number of clusters; insufficient flexibility due to reliance on preset or fixed assumptions; high communication overhead, frequent exchange of complete model parameters, or reliance on complex data calculations, affecting training efficiency and scalability; and difficulty in balancing data privacy with model performance and computational efficiency, resulting in limited personalized learning effects and slow convergence speed.

[0005] To achieve the above objectives, the first aspect of this application provides a federated learning data processing method, the method comprising: Distribute the initial global model to multiple clients; Receive model gradient information uploaded by each client after training the initial global model based on its local dataset; Multiple candidate clustering schemes are randomly generated, and each candidate clustering scheme corresponds to a grouping method that divides all clients into different clusters; For each candidate clustering scheme, the client set within each cluster is determined according to the candidate clustering scheme, the aggregation gradient of each cluster is calculated, and the evaluation index of the candidate clustering scheme is determined based on the aggregation gradient of all clusters and the independent dataset used for model evaluation. The independent dataset is a dataset that is independent of all clients and can be called by all clients. The independent dataset includes multiple sample features and the classification category corresponding to each sample feature. The target user clustering scheme is determined from multiple candidate clustering schemes based on evaluation indicators; Clients are grouped based on a target user clustering scheme, and model aggregation training is performed for each group to obtain the final model corresponding to each group.

[0006] In the embodiments of this application, the evaluation metrics include accuracy and entropy. Accuracy is used to reflect the predictive performance of the model, and entropy is used to measure the uncertainty of the model.

[0007] In this embodiment, the accuracy is determined according to the following formula:

[0008] Where k is the number of clusters, the number of correctly predicted samples refers to the number of samples correctly classified by the model on the independent dataset, and the total number of samples is the total number of samples in the independent dataset.

[0009] In this embodiment, the entropy value is determined according to the following formula:

[0010] Where k is the k-th cluster, It is the model's first in the independent dataset. The probability of a sample being predicted as belonging to a certain category. Used to quantify prediction accuracy It is the number of samples in the cluster that validate the independent dataset.

[0011] In this embodiment, a scoring function is used to evaluate the performance of each cluster. The expression of the scoring function is as follows:

[0012] Where λ is the weighting coefficient. It's about accuracy. It is the entropy value.

[0013] In the embodiments of this application, the method further includes: after selecting the candidate clustering scheme with the best performance in each iteration, using an iterative update and cross-generation mechanism to obtain a new candidate clustering scheme.

[0014] In this embodiment of the application, the cross-generation mechanism includes: taking the two candidate clustering schemes with the best performance evaluation index in this iteration as parent candidate clustering schemes, and calculating the Hamming distance between the two parent candidate clustering schemes; generating a child candidate clustering scheme based on the two parent candidate clustering schemes and the Hamming distance between the two parent candidate clustering schemes; wherein the Hamming distance between the child candidate clustering scheme and the two parent candidate clustering schemes is equal.

[0015] A second aspect of this application provides a federated learning data processing apparatus, comprising: a memory configured to store instructions; and a processor configured to retrieve instructions from the memory and, when executing the instructions, to implement any of the aforementioned federated learning data processing methods.

[0016] A third aspect of this application provides a machine-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform any of the above-described federated learning data processing methods.

[0017] The fourth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described federated learning data processing methods.

[0018] This application proposes a federated learning data processing method. By optimizing user clustering strategies and utilizing model gradient information, it improves model performance and communication efficiency while ensuring data privacy and security, providing technical support for achieving broader cross-institutional data sharing and collaborative learning. This significantly reduces communication costs, improves model training efficiency, model accuracy and generalization ability, enhances the adaptability and robustness of the clustering process, and enables efficient collaborative learning without disclosing the original data.

[0019] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 The schematic diagram illustrates a flowchart of a federated learning data processing method according to an embodiment of this application; Figure 2 The schematic diagram illustrates a flowchart of a federated learning data processing method according to another embodiment of this application; Figure 3 This illustration schematically shows an architecture diagram of a medical diagnostic federated learning system according to an embodiment of this application; Figure 4The diagram illustrates the internal structure of a computer device according to an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] Figure 1 The illustration shows a flowchart of a federated learning data processing method according to an embodiment of this application. Figure 1 As shown in one embodiment of this application, a federated learning data processing method is provided, including the following steps: Step 101: Distribute the initial global model to multiple clients.

[0023] In this step, the server initializes the global model parameters θ(0). During initialization, the weight parameters of all layers of the model are randomly generated using a standard normal distribution N(0,1) to ensure that the initial state of training is unbiased. After the model parameters θ(0) are initialized, the server broadcasts them to all participating clients via a secure communication protocol. Specifically, HTTPS protocol based on TLS (Transport Layer Security) is used for encrypted transmission to ensure that the parameters are not tampered with during communication. All clients receive identical initial parameters. After receiving the model parameters, the clients store them locally to prepare for subsequent local training. Depending on the system configuration, training can be conducted in synchronous mode to ensure that all clients start simultaneously, or in asynchronous mode to improve the overall training speed. Throughout the training process, the server can periodically broadcast updated global models to clients as needed to ensure that each client always learns based on the latest model. This step ensures the integrity and consistency of the initial model through secure encrypted transmission, providing a reliable and synchronous starting point for subsequent distributed collaborative training.

[0024] Step 102: Receive the model gradient information uploaded by each client after training the initial global model based on its local dataset.

[0025] Assume there are n clients in the federated learning system, each client It will be based on its own local dataset Perform local training. Dataset Owned by user i It consists of 1 data point, each data point It is a vector containing input features and corresponding categories The distribution of these data is unknown to the central server. Specifically, the client uses the standard gradient descent algorithm to calculate the gradient of the model parameters by minimizing the loss function. The gradient information reflects the direction of model optimization under the client's data distribution. The gradient calculation expression is: .in, Indicates the model parameters The partial derivatives of the gradient. After completing the local gradient calculation, each client uploads the resulting gradient to the server. The server receives gradient information from all participating clients, and these gradients will be used for subsequent clustering analysis and global model updates.

[0026] This step enables the server to collect gradient information reflecting the data distribution characteristics of each client without obtaining the raw local data, providing a foundation for subsequent gradient-based clustering analysis and supporting efficient personalized model collaborative training while protecting user data privacy.

[0027] Step 103: Randomly generate multiple candidate clustering schemes, where each candidate clustering scheme corresponds to a grouping method that divides all clients into different clusters.

[0028] In this step, the server, based on a preset number of clusters K, generates np different client partitioning schemes within the grouping search space comprised of all clients using a random combination generation algorithm. np is typically set to 50 or 100 and can be adjusted according to the actual scenario. Each partitioning scheme constitutes a candidate clustering scheme, representing a specific grouping strategy for assigning all clients to K clusters. By generating completely randomly rather than traversing all possible combinations, the server can significantly reduce the computational complexity of the scheme generation stage while maintaining search breadth. This step, through a lightweight random generation mechanism, efficiently constructs a diverse initial grouping candidate set, providing a foundation for subsequent rapid evaluation and selection based on gradient information. It effectively avoids the communication and computational burden caused by complex similarity calculations or high-dimensional data exchange in traditional clustering methods, improving the algorithm's flexibility and scalability when handling non-independent and identically distributed data.

[0029] Step 104: For each candidate clustering scheme, determine the client set within each cluster according to the candidate clustering scheme, calculate the aggregation gradient of each cluster, and determine the evaluation index of the candidate clustering scheme based on the aggregation gradient of all clusters and the independent dataset used for model evaluation; wherein, the independent dataset is a dataset that is independent of all clients and can be called by all clients, and the independent dataset includes multiple sample features and the classification category corresponding to each sample feature.

[0030] For each candidate clustering scheme generated in step 103, the server first parses the scheme to determine the set of clients included in each cluster. For each cluster, the server calculates its aggregate gradient by taking a weighted average of the gradient information uploaded by all clients within the cluster, thus obtaining the aggregate gradient representing the overall update direction of the cluster. . ,in, The gradient uploaded by client i This represents the number of clients in the cluster. The server pre-stores an independent dataset for objectively evaluating each candidate clustering scheme. This dataset does not originate from the clients participating in federated learning, is independent of the local data of all clients, and does not contain any client-specific private data, thus ensuring data privacy and security, and guaranteeing the objectivity of the evaluation. This dataset is pre-collected and stored by the server and contains a number of sample features and their corresponding classification categories. Specifically, each sample contains an input feature vector. and corresponding categories The model is used for simulation inference and performance evaluation. Feature vectors are input into the updated model, which performs classification inference on the input feature vectors based on its internal parameters and outputs predicted categories. The classification categories corresponding to the feature vectors of the current sample in an independent dataset serve as supervision information, used to compare with the predicted categories output by the model. This dataset can be a publicly available standard dataset, validation data related to the target task domain collected by the server itself, or an independent test set extracted from historical data. The number of samples in the dataset can be set according to the actual application scenario, with a default setting of 200–1000 samples to ensure statistical significance of the evaluation while controlling computational costs. The server uses all obtained aggregated gradients to perform forward inference on the independent dataset used for model evaluation. For each cluster k, the server uses the aggregated gradients... Simulate and update the model: Where α is the preset global learning rate, For the model parameters of client i in round t, the server uses the simulated updated model. Forward inference is performed on all samples in the independent dataset to obtain the predicted probability distribution for each sample. Based on the inference results, the evaluation metric corresponding to the candidate clustering scheme is calculated. This step utilizes the independent dataset and aggregated gradients to quickly simulate and evaluate the candidate scheme. It can efficiently predict the potential performance and stability of different grouping strategies without the need for multiple rounds of real training, thus providing a reliable basis for selecting the optimal clustering scheme. This significantly reduces the huge communication and computational overhead caused by repeated trials of different groupings in traditional methods.

[0031] In one embodiment, the evaluation metrics include accuracy and entropy, where accuracy reflects the model's predictive performance and entropy measures the model's uncertainty.

[0032] Specifically, for each candidate clustering scheme, the server uses its aggregation gradient to perform inference on an independent evaluation dataset and calculates two key metrics based on the inference results: accuracy and entropy. Accuracy is calculated as the ratio of the number of samples correctly predicted by the model to the total number of samples in the dataset, used to quantify the model's classification performance. Entropy is calculated based on the class probability distribution output by the model for each sample; this value reflects the confidence level of the model's predictions, with lower entropy indicating more certain and stable predictions. This embodiment, by simultaneously examining accuracy and entropy, can more comprehensively evaluate the quality of candidate clustering schemes, avoiding evaluation biases that may result from relying on only a single performance metric, thus more reliably selecting the optimal grouping strategy that maintains high prediction accuracy while possessing good stability.

[0033] In one embodiment, accuracy is determined according to the following formula:

[0034] Where k is the number of clusters, the number of correctly predicted samples refers to the number of samples correctly classified by the model on the independent dataset, and the total number of samples is the total number of samples in the independent dataset. For each candidate clustering scheme, the server first uses the aggregate gradient calculated under that scheme to simulate and update the model, and then uses the updated model to perform inference on the independent dataset. By comparing the model's prediction results of sample features in the independent dataset with the correct classification category corresponding to that sample, the number of correctly classified samples is obtained, and this number is divided by the total number of samples in the dataset to obtain the accuracy index corresponding to that scheme. This method objectively evaluates the model's prediction performance through the above explicit quantitative formula, providing a unified and computable performance metric for horizontal comparison of different clustering schemes. This helps to accurately identify client grouping strategies that can effectively improve the model's classification ability, enhancing the reliability and interpretability of the scheme selection process.

[0035] In one embodiment, the entropy value is determined according to the following formula:

[0036] Where k is the k-th cluster, It is the model's first in the independent dataset. The probability of a sample being predicted as belonging to a certain category. Used to quantify prediction accuracy This represents the number of samples in the independent dataset used to validate the cluster. The independent dataset is pre-collected and stored by the server, containing multiple sample features and the corresponding classification category for each feature. For each candidate clustering scheme, the server updates the model using the corresponding aggregation gradient and then performs inference on the independent dataset to obtain the model's probability distribution for each sample's category. Based on this probability distribution, the sum of the entropy values ​​of all samples under that scheme is calculated, yielding the entropy index characterizing the model's prediction uncertainty. The lower the entropy value, the more certain and stable the model's prediction results for the data under that clustering scheme. This embodiment quantifies the confidence level of the model's predictions through information entropy, providing a calculable metric for evaluating the stability of clustering schemes. Combined with the accuracy index, it can more comprehensively screen for the optimal grouping strategy that maintains high prediction performance while possessing good reliability, enhancing the algorithm's robustness to noisy data and distribution shifts.

[0037] In one embodiment, a scoring function is used to evaluate the performance of each cluster. The expression for the scoring function is as follows:

[0038] Where λ is the weighting coefficient. It's about accuracy. It is the entropy value. For each candidate clustering scheme k, the server calculates the accuracy separately. and entropy Then, its comprehensive score is calculated based on the above scoring function. By adjusting The value of can flexibly control the relative importance of model prediction performance and prediction stability in the evaluation. Finally, the system ranks and selects based on the scores of each candidate scheme. This embodiment achieves joint optimization of model accuracy and confidence by introducing a scoring function with adjustable weights. This mechanism enables the algorithm to adaptively select the clustering scheme that achieves the best balance between performance and stability according to the actual task requirements, thereby improving the overall robustness and practicality of the federated learning system under complex data distributions.

[0039] Step 105: Determine the target user clustering scheme from multiple candidate clustering schemes based on the evaluation indicators.

[0040] The accuracy of each combination is calculated in step 104. and entropy Then, the server uses a comprehensive scoring function to evaluate the merits of each combination. Finally, the combination with high accuracy and low entropy is selected as the optimal combination for the model, i.e., the combination with the highest score. The highest combination This combination demonstrates strong model stability. The server will then use this optimal combination in subsequent training phases for local training and global model updates until the algorithm converges. In each iteration, the server generates multiple random user combinations. The accuracy and entropy of each combination are calculated. Based on the comprehensive score, the optimal combination is selected and used for the next round of training and model updates. If the optimal combination... If a change occurs, the server continues with the next iteration, generating new combinations and updating the global model; if the optimal combination... If the clustering remains unchanged, the current clustering scheme is considered to have converged. At this point, the server can check the model's convergence or increase the number of clusters to select a new optimal combination. If the optimal combination... If the algorithm remains unchanged throughout multiple iterations, it indicates convergence, and the server can stop iterating to confirm the final optimal combination. The final user grouping scheme will be determined at this point and used for the final training of the global model.

[0041] Step 106: Group the clients based on the target user clustering scheme, and perform model aggregation training for each group to obtain the final model corresponding to each group.

[0042] After model convergence, the server will use the optimal combination for final global model training. In this stage, each client will train locally based on its cluster group, calculate its local gradient, and upload it to the server via an encrypted protocol. The uploaded gradient information undergoes differential privacy processing to ensure the privacy and security of user data. Based on the gradient information uploaded by all clients, the server updates the global model parameters using the same aggregation method. The server then evaluates the performance of the global model using independent validation datasets to ensure its generalization ability across all user data. When the model evaluation results reach the preset performance target, the model is considered to have been trained and converged. The finally trained global model will be deployed to all clients for practical application, and the model can also be updated in real time based on new data.

[0043] In one embodiment, the method further includes: after selecting the candidate clustering scheme with the best performance in each iteration, using an iterative update and cross-generation mechanism to obtain new candidate clustering schemes. Specifically, after evaluating all candidate clustering schemes in the current round and determining the optimal scheme, the server performs iterative update and cross-generation operations based on the optimal scheme to construct a set of candidate schemes for the next iteration. Iterative update includes randomly adjusting the cluster labels of some clients in the optimal scheme; cross-generation refers to structurally combining the optimal scheme with other schemes with higher scores, such as swapping the clusters to which some clients belong, thereby generating a new scheme with some features of both. Through the above mechanism, the system can explore its neighboring solution space while retaining the current optimal solution, and introduce structural diversity to avoid the search getting trapped in local optima. This embodiment, by introducing an iterative update and cross-generation mechanism, enables the clustering search process to have continuous optimization and exploration capabilities, and can more efficiently approach the globally optimal grouping scheme within a limited number of iterations, thereby improving the overall clustering quality and the final performance of federated learning.

[0044] In one embodiment, the cross-generation mechanism includes: selecting the two candidate clustering schemes with the best performance evaluation index in this iteration as parent candidate clustering schemes, and calculating the Hamming distance between the two parent candidate clustering schemes; generating a child candidate clustering scheme based on the two parent candidate clustering schemes and the Hamming distance between the two parent candidate clustering schemes; wherein the Hamming distance between the child candidate clustering scheme and the two parent candidate clustering schemes is equal.

[0045] Specifically, the server selects the two candidate clustering schemes with the highest comprehensive scores from the current iteration round, designating them as parent scheme A and parent scheme B, respectively. By comparing the differences in their client cluster label assignment sequences, the server calculates their Hamming distance 'd', which represents the number of clients with different label values. Subsequently, based on this distance 'd', the server constructs a child scheme C, ensuring that its Hamming distance is equal to both scheme A and scheme B. This generation process can be achieved by performing a finite number of label replacements or reassignments at the differing positions of the parent schemes, ensuring that the child scheme both incorporates the structural features of its parents and maintains symmetrical differences with them. This embodiment, by introducing an equal Hamming distance crossover strategy, can systematically explore the structural balance point between the two while preserving excellent parent features, enhancing the guidance and diversity in the clustering scheme search process. This facilitates approximating a better grouping structure in fewer iteration rounds, improving overall optimization efficiency.

[0046] In one embodiment, after evaluating candidate schemes for the current number of clusters K, if the optimal cluster combination is... If the clustering scheme remains stable through multiple iterations, it is determined that the current clustering scheme has converged. At this point, the server increases the number of clusters K by 1 and, based on the new K value, re-executes step 103 to randomly generate np new candidate clustering schemes. For each candidate clustering scheme, the server calculates the average of the gradients of all clients within each cluster under that scheme, obtaining the aggregate gradient of each cluster. Specifically, for a cluster containing a client set... Clusters, their aggregation gradient for:

[0047] in, The gradient uploaded by client i This represents the number of clients in the cluster. The server updates the data based on the aggregation gradient of each cluster, using the following formula: The global model parameters are updated, where α is the preset global learning rate. Let K be the model parameters for client i in round t. After the update, the server returns the updated model parameters to the client for the next round of local training and gradient calculation. This process is repeated after each increase in the value of K until the optimal clustering scheme remains stable for multiple consecutive values ​​of K, or the model performance no longer significantly improves. This embodiment, by adaptively increasing the number of clusters, allows the system to dynamically explore more refined data grouping structures, thereby better adapting to the actual distribution differences of client data. This mechanism avoids the subjective limitation of preset cluster numbers, improves the algorithm's adaptability to different heterogeneous scenarios, and achieves continuous optimization of the global model through gradient aggregation updates.

[0048] In one embodiment, the client's local training parameters, learning rate The default value is set to 0.01; batch size The default batch size is set to 128; the maximum number of training epochs is [not specified]. The default value is set to 50. These parameters can be adjusted according to different application scenarios; for example, with large datasets, the batch size may need to be adjusted. or learning rate To balance training speed and accuracy, the effectiveness of the pre-iterative federated clustering algorithm of this invention was verified. Experiments were conducted on the standard dataset Fashion-MNIST (FMNIST), simulating 100 clients and including non-independent identically distributed (IID) data, and compared with existing baseline methods. This experiment used several publicly available and widely used benchmark datasets to simulate different types of IID scenarios. To simulate a real federated learning environment, data for each client was randomly assigned, and different intensities (e.g., 10%, 30%, 50%) of label distribution skew and number skew were artificially constructed for Non-IID data partitioning by controlling the number of classes and samples per client. Experimental results show that under various Non-IID settings, this invention significantly outperforms the baseline in model accuracy. For example, in the 30% data skew scenario, compared to the optimal baseline, the accuracy of this invention's model is improved by an average of over 5.63%, and the number of communication rounds required to reach convergence is reduced by approximately 35%. The results fully demonstrate that by optimizing the user clustering strategy, this invention significantly improves model performance while effectively reducing communication overhead, successfully solving the technical problem of existing federated clustering techniques struggling to balance performance, cost, and flexibility. Table 1 below compares the performance of the proposed method with existing baseline methods on the FMNIST dataset under different degrees of non-independent and identically distributed characteristics.

[0049] Table 1

[0050] As shown in Table 1, under three different levels of Non-IID settings, the method of this invention significantly outperforms IFCA and PACFL in classification accuracy, especially in the 20% Non-IID scenario where the accuracy improvement is most significant. Meanwhile, the method exhibits stable and low recall, indicating that it maintains high accuracy without sacrificing recall, demonstrating its good balance and stability in heterogeneous data environments.

[0051] As shown in Table 2 below, Table 2 compares the total number of communication rounds required for different methods to achieve convergence on the FMNIST dataset under different degrees of non-independent and identically distributed characteristics.

[0052] Table 2

[0053] As shown in Table 2, the communication efficiency advantage of the proposed method becomes increasingly apparent with the increase in the degree of Non-IID. Under the 20% and 30% Non-IID settings, the proposed method converges in only 10 and 11 rounds of communication, respectively, significantly lower than IFCA and PACFL. In particular, the communication cost is reduced by approximately 35% compared to IFCA in the 30% scenario. This indicates that the proposed method, through pre-iterative clustering and gradient similarity search mechanisms, effectively reduces the communication burden between the client and server while ensuring model performance, making it more suitable for practical federated learning scenarios with limited communication.

[0054] Figure 1 This is a flowchart illustrating a federated learning data processing method in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0055] In one embodiment, Figure 2The diagram illustrates a flowchart of a federated learning data processing method according to another embodiment of this application. As shown, the server constructs a global machine learning model and randomly initializes its parameters, then securely distributes the initial model to each participating client. Each client trains the received model based on its local dataset, calculating the gradient information of the model parameters using a gradient descent algorithm. Each client encrypts the locally calculated gradient information and uploads it to the server; no original local data is shared throughout the process. Based on the received gradient information, the server randomly generates multiple candidate user clustering schemes, each scheme corresponding to a grouping method that assigns clients to different clusters. For each candidate clustering scheme, the server calculates the gradient information for each cluster. The algorithm aggregates gradients and uses an independent evaluation dataset for simulated inference to calculate the prediction accuracy and information entropy of the proposed scheme. Based on the accuracy and entropy values ​​of each candidate scheme, the server uses a scoring function to select the current optimal clustering scheme and updates the model accordingly, generating a new round of combinations, forming an iterative optimization process. If the optimal clustering scheme remains stable across multiple iterations, the clustering process is considered converged; otherwise, it returns to the previous step to continue iterating. Based on the final determined clustering scheme, the client is grouped, and federated learning training is performed on each group until the models of each group converge, resulting in a final model that matches the data distribution of each group. This flowchart clearly illustrates the entire process of this method from model initialization, gradient uploading, cluster search, iterative optimization to final group training, demonstrating its technical path of achieving efficient and personalized federated learning while protecting data privacy.

[0056] In one embodiment, Figure 3This diagram schematically illustrates the architecture of a federated learning system for medical diagnosis according to an embodiment of this application. The federated learning system includes a central server and multiple medical institutions as clients: Hospital A, Hospital B, and Hospital C. Each hospital holds its medical data locally and does not directly share raw data with the server or other hospitals. For example, the federated learning task in this embodiment is to determine whether a patient has heart disease based on their physiological indicators. The classification categories include: having heart disease and not having heart disease. Due to differences in patient groups among different hospitals, the category distribution of the local data in each hospital exhibits non-independent and identically distributed characteristics. For example, Hospital A is located in the city center, treating mainly middle-aged and elderly patients; in its local data, positive samples account for approximately 70%, and negative samples account for approximately 30%. Hospital B is located in a community clinic, treating mostly patients with mild symptoms or those undergoing routine checkups; in its local data, negative samples account for approximately 80%, and positive samples account for approximately 20%. Hospital C is located in a specialized cardiovascular hospital, treating mostly confirmed or suspected heart disease patients; in its local data, positive samples account for approximately 90%, and negative samples account for approximately 10%. The differences in data distribution make it difficult for a single global model to simultaneously adapt to the needs of three hospitals. Therefore, clustering is needed to train personalized models for hospitals with similar data distributions. The server pre-stores an independent public dataset for objectively evaluating candidate clustering schemes. In this embodiment, this independent dataset can be 1000 standardized cardiac patient data points obtained by the server from public channels. Each data point includes various physiological indicators of the patient—i.e., sample features—and the actual diagnosis of whether or not the patient has cardiac disease—i.e., the classification category. This dataset is independent of the local data of Hospital A, Hospital B, and Hospital C, and does not contain any hospital's private patient information, thus avoiding privacy risks. In each iteration, the server randomly generates multiple candidate clustering schemes based on the gradient information uploaded by each hospital. For each candidate scheme, the server calculates the aggregate gradient for each hospital within each group, then uses this aggregate gradient to simulate and update the current model, and performs forward inference on the 1000 independent public datasets. The server calculates a comprehensive score for each candidate scheme based on accuracy and entropy, selecting the scheme with the highest score as the target clustering scheme. Subsequently, the server performs formal federated learning training according to this grouping strategy. During the formal training phase, the server distributes the model parameters corresponding to each group to the clients within that group. The clients train based on their local data and upload gradients or parameter updates. The server then aggregates the parameters according to the grouping strategy and updates the corresponding cluster model. This embodiment, through encrypted communication and parameter aggregation mechanisms, achieves effective collaborative utilization of cross-institutional medical data while fully protecting the data privacy of each hospital, providing a feasible technical path for building high-performance, generalizable medical intelligent models.

[0057] In one embodiment, a federated learning data processing apparatus (not shown) is provided, comprising: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and, when executing instructions, to implement any of the aforementioned federated learning data processing methods.

[0058] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and the federated learning data processing method can be implemented by adjusting kernel parameters.

[0059] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0060] This application provides a storage medium on which a program is stored, which, when executed by a processor, implements the above-described federated learning data processing method.

[0061] This application provides a processor for running a program, wherein the program executes the above-described federated learning data processing method during runtime.

[0062] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The network interface A02 is used for communication with external terminals via a network connection. When executed by the processor A01, the computer program B02 implements a federated learning data processing method.

[0063] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0064] This application provides a computer (electronic) device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the above federated learning data processing methods.

[0065] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform the steps of initializing a federated learning data processing method.

[0066] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0069] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0070] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0071] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0072] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0073] It should also be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0074] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A federated learning data processing method, characterized in that, include: Distribute the initial global model to multiple clients; Receive model gradient information uploaded by each client after training the initial global model based on its local dataset; Multiple candidate clustering schemes are randomly generated, wherein each candidate clustering scheme corresponds to a grouping method that divides all clients into different clusters; For each candidate clustering scheme, the client set within each cluster is determined according to the candidate clustering scheme, the aggregation gradient of each cluster is calculated, and the evaluation index of the candidate clustering scheme is determined based on the aggregation gradient of all clusters and the independent dataset used for model evaluation; wherein, the independent dataset is a dataset that is independent of all clients and can be called by all clients, and the independent dataset includes multiple sample features and the classification category corresponding to each sample feature; The target user clustering scheme is determined from multiple candidate clustering schemes based on the evaluation indicators. The clients are grouped based on the target user clustering scheme, and model aggregation training is performed for each group to obtain the final model corresponding to each group.

2. The method according to claim 1, characterized in that, The evaluation metrics include accuracy and entropy. Accuracy reflects the predictive performance of the model, and entropy measures the uncertainty of the model.

3. The method according to claim 2, characterized in that, The accuracy is determined according to the following formula: Where k is the number of clusters, the number of correctly predicted samples refers to the number of samples correctly classified by the model on the independent dataset, and the total number of samples is the total number of samples in the independent dataset.

4. The method according to claim 2, characterized in that, The entropy value is determined according to the following formula: Where k is the k-th cluster, It is the model's first in the independent dataset. The probability of a sample being predicted as belonging to a certain category. Used to quantify prediction accuracy It is the number of samples in the cluster that validate the independent dataset.

5. The method according to claim 2, characterized in that, A scoring function is used to evaluate the performance of each cluster, and the expression for the scoring function is as follows: Where λ is the weighting coefficient. It's about accuracy. It is the entropy value.

6. The method according to claim 1, characterized in that, The method further includes: After selecting the candidate clustering scheme with the best performance in each iteration, a new candidate clustering scheme is obtained by using an iterative update and cross-generation mechanism.

7. The method according to claim 6, characterized in that, The cross-generation mechanism includes: The two candidate clustering schemes with the best performance evaluation index in this iteration are selected as parent candidate clustering schemes, and the Hamming distance between the two parent candidate clustering schemes is calculated. A child candidate clustering scheme is generated based on two parent candidate clustering schemes and the Hamming distance between the two parent candidate clustering schemes; wherein the Hamming distance between the child candidate clustering scheme and the two parent candidate clustering schemes is equal.

8. A federated learning data processing device, characterized in that, include: The memory is configured to store instructions; A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the federated learning data processing method according to any one of claims 1 to 7.

9. A machine-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, this instruction causes the processor to be configured to perform the federated learning data processing method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the federated learning data processing method according to any one of claims 1 to 7.

Images