A federated learning method based on multi-target client selection and moro envelope regularization

Abstract

The application relates to the technical field of distributed machine learning, and discloses a federated learning method based on multi-target client selection and Moro envelope regularization, which comprises the following steps: S1, preprocessing and heterogeneous scene construction; S2, multi-target client selection: in each round of training, the data quality, model performance, resource efficiency and system fairness of each client are comprehensively evaluated, the optimal client subset is screened through a non-dominated sorting algorithm after weighted fusion; S3, regularization model training: the client performs local training based on a loss function comprising a local and global Moro envelope regularization term, and uploads parameters; S4, knowledge distillation and aggregation: the server generates a global knowledge representation, calculates a similarity weight based on model consistency, and updates a global model through weighted aggregation; and S5, iterative optimization: repeating S2-S4 until convergence, and outputting a global model and a personalized local model. Through multi-target intelligent client selection and regularization anti-drift, the application effectively improves the precision, speed and fairness of federated learning.

Description

Technical Field

[0001] This invention belongs to the field of distributed machine learning technology, and specifically relates to a federated learning method based on multi-objective client selection and Moro envelope regularization. Background Technology

[0002] With the rapid development of mobile internet and IoT technologies, and the increasingly stringent data privacy regulations worldwide, how to achieve collaborative machine learning across devices and institutions while protecting user data privacy has become a core issue of common concern for both academia and industry. Federated learning, as an emerging distributed machine learning paradigm, allows multiple clients to collaboratively train a global model without sharing raw local data, thus demonstrating significant application value in scenarios such as smart healthcare, financial risk control, and intelligent IoT.

[0003] However, in practical deployments of federated learning systems, client data often exhibits typical non-independent and identically distributed (NID) characteristics, meaning that data from different clients differs significantly in terms of class distribution, sample size, and feature space. While existing federated learning methods have made some progress in dealing with NID data, they still have significant shortcomings in large-scale, heterogeneous deployment scenarios. In particular, insufficient comprehensive evaluation of client value, limited dynamic suppression of model drift, and an imperfect strategy for balancing global generalization and local personalization severely restrict the reliable application of federated learning in key domains.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To address or at least mitigate one or more of the above problems, a federated learning method based on multi-objective client selection and Moro envelope regularization is provided.

[0006] To achieve the above objectives, according to a first aspect of the present invention, a federated learning method based on multi-objective client selection and Moro envelope regularization is provided, comprising: S1. Data Preprocessing and Heterogeneous Scene Construction: Preprocess the private data stored in a decentralized manner on each client and build a federated learning heterogeneous data environment; S2, Multi-target client selection: For each round of federated training, the data quality score, model performance score, resource efficiency score, and system fairness score of each client are evaluated; a non-dominated ranking algorithm and crowding distance calculation are used to select the Pareto optimal client subset from all clients; S3. Construct and train a federated learning model that incorporates Moro envelope regularization; The selected client trains the local model using a loss function that includes a local Moro envelope regularization term and a global Moro envelope regularization term; wherein the local Moro envelope regularization term is used to constrain the deviation between the local model parameters and the global model parameters, and the global Moro envelope regularization term is used to constrain the stability of the global model parameter updates. Each client uploads the trained local model parameters to the server; S4. Knowledge distillation and similarity-weighted aggregation: The server generates a global knowledge representation based on the local models uploaded by each client, calculates similarity weights based on the consistency between each local model and the global model, and uses the similarity weights to perform weighted aggregation of local model parameters to update the global model. S5. Iteration and Output: Repeat steps S2 through S4 until the preset convergence condition is met, and output the final global model and the personalized local model for each client.

[0007] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art: This invention combines multi-objective optimization of client selection with Moro envelope regularization to systematically balance multiple objectives such as model accuracy, training efficiency, resource consumption, and system fairness. The multi-objective selection mechanism dynamically evaluates the comprehensive value of clients, replacing traditional random or single-metric selection strategies. This allows for the selection of a subset of clients that contributes most to the global model and maintains system health in each training round, fundamentally improving the overall efficiency and robustness of federated learning.

[0008] The adaptive Moro envelope regularization framework proposed in this invention effectively addresses the core challenges of model drift and convergence instability caused by non-independent and identically distributed (Non-IID) data. This regularization term applies smoothing constraints during both the client-side local training and server-side global aggregation phases, ensuring that personalized models adapt to local data characteristics without excessively deviating from global consensus. Simultaneously, the global model can more stably integrate knowledge from heterogeneous clients, thereby simultaneously improving both the model's generalization ability and personalized performance.

[0009] To further optimize knowledge transfer efficiency, this invention deeply integrates Moro envelope regularization with a knowledge distillation mechanism. This integration creates the dual advantages of explicit parameter constraints and implicit knowledge guidance, enabling clients to efficiently absorb global knowledge through soft label propagation and regularization guidance without sharing original data or even public datasets. This significantly enhances the collaborative performance and convergence speed of the model in extremely non-independent and identically distributed (Non-IID) scenarios.

[0010] The end-to-end Regularization-AwareClient Selection Federated Learning (RACS-FL) framework constructed in this invention has a core advantage in achieving a closed-loop linkage between intelligent client selection and regularization-aware aggregation. The selection mechanism provides a high-quality source of model updates for subsequent aggregation, while the model state generated during the regularization aggregation process, in turn, provides an evaluation basis for the next round of selection. This synergistic effect enables the entire federated learning system to adapt to changes in data distribution and client state, ultimately achieving significant comprehensive benefits on multiple benchmark datasets, including improved average accuracy, faster convergence speed, and enhanced fairness index. The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0011] The accompanying drawings, as part of this invention, are provided to further illustrate the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention, but do not constitute an undue limitation thereof. Clearly, the drawings described below are merely some embodiments; those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0012] In the attached diagram: Figure 1 This is a flowchart illustrating the federated learning method based on multi-objective client selection and Moro envelope regularization in this specific embodiment. Figure 2 This is a flowchart illustrating the multi-target client selection process in this specific embodiment; Figure 3 This is a schematic diagram of the Moro envelope regularization process in this specific embodiment. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of protection of the present invention.

[0014] Please see Figure 1 This invention provides a federated learning method based on multi-objective client selection and Moro envelope regularization, comprising: S1. Data Preprocessing and Heterogeneous Scene Construction: Preprocess the private data stored in a decentralized manner on each client and build a federated learning heterogeneous data environment; S2, Multi-target client selection: For each round of federated training, the data quality score, model performance score, resource efficiency score, and system fairness score of each client are evaluated; a non-dominated ranking algorithm and crowding distance calculation are used to select the Pareto optimal client subset from all clients; S3. Construct and train a federated learning model that incorporates Moro envelope regularization; The selected client trains the local model using a loss function that includes a local Moro envelope regularization term and a global Moro envelope regularization term; wherein the local Moro envelope regularization term is used to constrain the deviation between the local model parameters and the global model parameters, and the global Moro envelope regularization term is used to constrain the stability of the global model parameter updates. Each client uploads the trained local model parameters to the server; S4. Knowledge distillation and similarity-weighted aggregation: The server generates a global knowledge representation based on the local models uploaded by each client, calculates similarity weights based on the consistency between each local model and the global model, and uses the similarity weights to perform weighted aggregation of local model parameters to update the global model. S5. Iteration and Output: Repeat steps S2 through S4 until the preset convergence condition is met, and output the final global model and the personalized local model for each client.

[0015] This invention combines a multi-objective client selection strategy with a Moro envelope regularization mechanism to achieve synergistic optimization between the generalization capability of the global model and the personalized needs of the client, while protecting the privacy of the client's original data. The complete process is as follows: Figure 1 As shown, the specific steps include:

[0016] Step 1: Data Preprocessing and Heterogeneous Scene Construction The private data stored separately on each client is preprocessed, including data cleaning, missing value detection, outlier identification, format standardization, feature normalization, and label encoding, to ensure that the local data on each client meets the requirements for model training.

[0017] To simulate data heterogeneity in real-world federated learning scenarios, a Dirichlet distribution is used to partition the benchmark dataset, and the degree of data heterogeneity is controlled by adjusting the distribution parameter α. The smaller α is, the more significant the differences in data category distributions among clients, and the higher the degree of data heterogeneity; the larger α is, the closer the data distribution among clients is to an independent and identically distributed (IID) state. For each data category, a K-dimensional allocation ratio vector is generated, and samples are allocated to K clients according to this ratio, so that the data from each client exhibits typical non-independent and identically distributed (Non-IID) characteristics such as imbalanced category distribution, differences in sample size, and differences in feature distribution.

[0018] Step 2: Selecting multiple target clients: To overcome the limitations of traditional client selection strategies that rely solely on random selection or single-metric evaluation, this embodiment designs a client selection mechanism based on multi-objective optimization. It comprehensively evaluates candidate clients from four dimensions: data quality, model performance, resource efficiency, and fairness, and selects a subset of clients to participate in the current round of training through non-dominated ranking and crowding distance calculation.

[0019] Data quality assessment measures the effectiveness of client-side local data for model training. This score comprehensively considers data diversity, data distribution consistency, data integrity, and outlier constraints to obtain the client-side... Data quality score .

[0020] Model performance evaluation measures the contribution of the client's historical training results to the global model. This score is calculated based on the client's historical training accuracy, training loss, and convergence stability. Model performance score .

[0021] Resource efficiency assessment measures the computational, communication, and energy requirements for a client to participate in the current training round. This score comprehensively considers computational power, network bandwidth, remaining energy, and training communication latency to determine the client's performance. Resource efficiency score .

[0022] Fairness assessment measures the balance of client participation in training over a long period. This score is calculated based on the client's historical participation rate, historical selection frequency, and model contribution. Fairness score .

[0023] In one alternative embodiment, the client The data quality score is expressed as follows: ; in, Indicates the client Data quality score, Indicates data diversity score, Indicates the consistency score of data distribution. Indicates data integrity score, Indicates the constraint score for outlier samples. These represent the weight coefficients of the corresponding sub-ratings, and .

[0024] Data diversity score is calculated using category distribution entropy and is expressed as follows: ; in, Indicates the client Data diversity score Indicates the client Categories in the data The proportion of samples, Indicates the total number of categories. This represents a minimal constant that avoids log(0).

[0025] Data distribution consistency score is obtained by comparing the statistical characteristics of client data and global data, and is expressed as: ; in, Indicates the client Data distribution consistency score, They represent the client respectively. The mean and standard deviation of the data These represent the mean and standard deviation of the global data, respectively.

[0026] Data integrity score is expressed as: ; in, Indicates the client Data integrity score Indicates the client The number of samples with missing fields, empty field values, or default field values. Indicates the client The total number of samples.

[0027] The outlier sample constraint score is represented as follows: ; in, Indicates the client Abnormal sample constraint scoring, Indicates the client The number of abnormal samples whose field values ​​do not meet the preset valid range or meet the preset anomaly detection rules.

[0028] The model performance score is expressed as follows: ; in, Indicates the client Model performance score This represents the normalized historical classification accuracy. This represents the normalized historical training loss. Indicates the client The convergence stability score, These represent the weight coefficients of the corresponding sub-indicators, and .

[0029] Resource efficiency score is expressed as: ; in, Indicates the client Resource efficiency score, This represents the normalized computational power. This represents the normalized network bandwidth. This represents the remaining energy after normalization. This represents the normalized training or communication latency. These represent the weight coefficients of the corresponding sub-indicators, and .

[0030] Fairness scores are expressed as follows: ; in, Indicates the client Fairness score, This indicates the number of times the normalized history was selected. This represents the normalized historical participation frequency. This represents the normalized model contribution. These represent the weight coefficients of the corresponding sub-indicators, and .

[0031] The server scores data quality. Model performance scoring Resource efficiency score And fairness score Build client The multi-objective evaluation vector is represented as: ; For any two clients a and b, if client a has a score no lower than client b on all target scores and a score higher than client b on at least one target score, then client a is considered to dominate client b. The server counts the number of times each client is dominated by other clients, and assigns clients not dominated by any client to the first front layer (Front_1). After removing clients and their dominance relationships from Front_1, the remaining clients are sorted by non-dominance, resulting in Front_2, Front_3, and so on, until all clients are assigned to their corresponding front layers.

[0032] Within the same frontier layer, the server sorts clients according to their target scores and calculates client performance. The crowding distance is expressed as: ; in, Indicates the client Crowded distance, and These represent the client under the target j sorting. The target values ​​of two adjacent clients, and Let m represent the maximum and minimum values ​​of target j in the current frontier layer, respectively, and m represent the number of targets. This represents a very small constant that avoids a denominator of zero.

[0033] The server selects clients sequentially based on the priority of the front layer. If the number of clients in a front layer does not exceed the preset selection limit after all clients have joined, all clients in that front layer are added to the candidate set. If the number of clients in a front layer exceeds the preset selection limit, clients within that front layer are selected in descending order of crowding distance until the preset selection limit is reached. This yields the Pareto optimal client subset for the current training round.

[0034] In an optional embodiment, a comprehensive client score can also be calculated as an auxiliary ranking criterion under the same frontier layer and the same congestion distance condition: ; in, Indicates the client Overall score represent the fusion weight coefficients corresponding to the data quality score, model performance score, resource efficiency score, and fairness score, respectively. .

[0035] Step 3: Local model training with Moro envelope regularization: The server distributes the current global model parameters to the selected clients. Each client initializes its local model with the received global model parameters and trains it based on its local private data. To mitigate the local model drift problem caused by non-independent and identically distributed data, a Moro envelope regularization term is introduced into the local training objective to constrain the offset between the local model parameters and the global model parameters.

[0036] In an optional embodiment, the local Moro envelope regularization term is expressed as: ; in, This represents the client's local model parameters. Indicates the current global model parameters. This represents the local Moro envelope regularization coefficient.

[0037] The global Moro envelope regularization term is represented as: ; in, Indicates the global model parameters for the current round. This represents the parameters of the global model in the previous round. This represents the global Moro envelope regularization coefficient.

[0038] The overall optimization objective of local training is expressed as: ; in, Represents cross-entropy loss, This represents the loss from knowledge distillation. Indicates feature alignment loss, This represents the local Moro envelope regularization term. This represents the global Moro envelope regularization term. Indicates the client Local model parameters, This indicates distillation-related parameters. These represent the weighting coefficients of the corresponding loss terms.

[0039] During local model training, a min-max alternating optimization strategy is employed. The client inputs the current training batch data into the local model to obtain the predicted output, reconstructed output, sequence association matrix, and prior association matrix, and calculates the cross-entropy loss, reconstruction loss, knowledge distillation loss, feature alignment loss, and association difference loss accordingly.

[0040] In the minimization phase, the parameters corresponding to the sequence association matrix are fixed, and the parameters corresponding to the local model parameters and the prior association matrix are updated to minimize the overall training loss. In the maximization phase, the parameters corresponding to the prior association matrix are fixed, and the parameters corresponding to the sequence association matrix are updated to increase the difference between the sequence association matrix and the prior association matrix. By alternately executing the minimization and maximization phases, the local model parameters trained on the current client are obtained. After training is complete, the client uploads the local model parameters, local prediction results, and local feature representations to the server.

[0041] In one embodiment, a min-max alternating optimization strategy is employed during local model training. This involves updating the model parameters by alternately executing a minimization update phase and a maximization update phase, specifically including the following steps: S31. The server sends the current global model parameters to the selected client, and the client initializes its local model parameters with the global model parameters. And sample the current training batch data from the local training data.

[0042] S32. The client inputs the current training batch data into the local model to obtain the reconstruction output, prediction output, sequence association matrix and prior association matrix. It then calculates the reconstruction loss based on the reconstruction output and the original input, the cross-entropy loss based on the prediction output and the true label, and the association difference loss based on the sequence association matrix and the prior association matrix.

[0043] S33. In the minimization update phase, the parameters corresponding to the sequence correlation matrix are fixed, and the parameters corresponding to the local model parameters and the prior correlation matrix are updated to minimize the comprehensive training loss. The comprehensive training loss is expressed as: ; in, This represents the overall training loss during the minimization phase. Represents cross-entropy loss, This represents the loss from knowledge distillation. Indicates feature alignment loss, This represents the local Moro envelope regularization term. This represents the global Moro envelope regularization term. Indicates the reconstruction loss. Indicates related difference loss, These represent the weighting coefficients of the corresponding loss terms; The local Moro envelope regularization term is expressed as: ; Wherein, it represents the client. In the The local model parameters of the wheel, Indicates the first Round global model parameters, Represents the local Moro envelope regularization coefficient; The global Moro envelope regularization term is expressed as: ; in, Indicates the first Round global model parameters, Indicates the first Round global model parameters, This represents the global Moro envelope regularization coefficient.

[0044] S34. In the maximization update phase, the parameters corresponding to the prior association matrix are fixed, and the parameters corresponding to the sequence association matrix are updated to increase the association difference between the sequence association matrix and the prior association matrix. The optimization objective in the maximization phase is expressed as: ; in, This represents the comprehensive training loss during the maximization phase. By introducing a negative correlation difference term into the loss function, the model increases the difference between the sequence correlation matrix and the prior correlation matrix when updating the sequence correlation parameters.

[0045] S35. Repeat the minimum update phase and the maximum update phase according to the preset number of alternating training cycles until the local training rounds are reached or the local convergence condition is met, and obtain the local model parameters trained on the current client.

[0046] S36. The client uploads the trained local model parameters, training loss, prediction results or soft labels to the server so that the server can perform similarity weighted aggregation and the next round of global model update.

[0047] Step 4: Knowledge distillation and similarity-weighted aggregation: S41. The server receives the local model parameters, local prediction results, and local feature representation uploaded by the selected client in the current round. The client... The uploaded local model parameters are denoted as The local prediction result is denoted as Local features are denoted as .

[0048] S42. The server generates a global knowledge representation based on the initial aggregation weights, which are represented as follows: ; in, Indicates the client The initial aggregation weights, Indicates the client The number of local samples, Indicates the first The set of clients selected in the round. The index number of the client set. This indicates the client's index number.

[0049] The global soft prediction result and global feature representation are generated based on the initial aggregate weights, as follows: ; ; in, This represents the global soft prediction result. This represents the global feature representation.

[0050] S43, Server-side computing client The consistency similarity between the representation and the global knowledge representation is expressed as: ; in, Indicates the client Consistency similarity, This represents the cosine similarity function.

[0051] S44. The server calculates client consistency similarity based on the client. The normalized similarity weights are expressed as: ; in, Represents the temperature coefficient, and represents the client set. Consistency similarity, Indicates the client The normalized similarity weights, and satisfying .

[0052] S45. The server performs weighted aggregation of the local model parameters uploaded by each client based on the normalized similarity weights to obtain the intermediate global model parameters, represented as: ; in, This represents the intermediate global model parameters obtained by weighted aggregation of normalized similarity weights. Indicates the client Normalized similarity weights, Indicates the client In the The local model parameters uploaded after local training. Indicates the first The set of clients selected in the round.

[0053] S46. The server modifies the intermediate global model parameters by combining the global Moro envelope regularization term, and obtains the first... The global model parameters are represented as follows: ; in, This represents the first term after modification by the global Moro envelope regularization term. Round global model parameters, This represents the intermediate global model parameters obtained from weighted aggregation. Indicates the first Round global model parameters, This represents the global Moro envelope regularization coefficient.

[0054] S47, The server will be updated to the next... The global model parameters are then distributed to the selected clients in the next round for subsequent local model training.

[0055] Step 5: Iterative Training and Model Output: Repeat steps 2 through 4 until a preset convergence condition is met. The preset convergence condition includes at least one of the following: reaching the maximum number of communication rounds, the global model validation loss changing less than a preset threshold for several consecutive rounds, and the global model accuracy reaching a preset target value.

[0056] After training, the final global model and the personalized local models for each client are output. The server can evaluate the generalization performance of the global model, the personalized performance of the client-side local models, the model convergence speed, communication efficiency, and the fairness of client selection based on the test set or validation set, providing a basis for actual federated learning deployment.

[0057] In an optional embodiment, when the method of the present invention is applied to a scenario of time-series anomaly detection or industrial sensor data anomaly early warning, the anomaly judgment threshold can be automatically determined during the inference phase using the anomaly score distribution of normal samples from the training phase. During testing, the data to be detected is input into the trained model to obtain the reconstruction output, sequence correlation matrix, and prior correlation matrix, and the reconstruction error and correlation difference for each time step are calculated respectively. For each time step... Its abnormal score is expressed as: ; in, Indicates the first Abnormal scores at each time step Indicates the input data at the [number]th [time]. Feature vectors at each time step The model represents the first The reconstruction results of the input features at each time step Represents the prior correlation matrix. Represents the sequence incidence matrix. This represents the difference in association calculated from the prior association matrix and the sequential association matrix. Indicates the first The association difference weights corresponding to each time step Indicates the first The reconstruction error at each time step.

[0058] Let Ω be the set of abnormal scores obtained from normal samples during the training phase, then the abnormality threshold is... It can be represented as: ; in, This represents the mean of the abnormal scores among normal samples during the training phase. This represents the standard deviation of outlier scores among normal samples during the training phase. This represents the preset threshold adjustment coefficient. When the test sample is at time step... abnormal scores Greater than the threshold When the time step is reached, the data corresponding to that time step is determined to be abnormal, and an abnormality warning is triggered.

[0059] Performance evaluation: Evaluating model performance from multiple dimensions provides a basis for model optimization and practical deployment.

[0060] Key evaluation metrics include average accuracy (ACC), convergence speed (number of communication rounds required to achieve stable performance), fairness index (performance difference coefficient among clients), and communication efficiency (communication cost per unit performance improvement). The benchmark selection includes mainstream federated learning algorithms, encompassing standard aggregation methods, knowledge distillation methods, and adaptive optimization methods: Standard aggregation methods include Federated Averaging (FedAvg) and Federated Proximal (FedProx); knowledge distillation methods include Federated Hyper-Knowledge Distillation (FedHKD) and Federated Model Distillation (FedMD); and the adaptive optimization method is Federated Learning with Adaptive Weight Aggregation (FedAWA). Heterogeneous scenario testing: The model was tested under different data heterogeneity levels (α=0.2~1.0) and different numbers of clients (10, 20, 30) to verify its robustness and scalability.

[0061] This embodiment targets scenarios such as Industrial Internet of Things (IIoT), medical data collaboration, and financial risk prediction. It uses publicly available benchmark datasets to verify the effectiveness of the Regularization-Aware Client Selection Federated Learning (RACS-FL) method. The specific settings are as follows: (1) Dataset selection: Five typical federated learning benchmark datasets were selected, covering image classification and structured data scenarios, as follows: CIFAR-10 dataset: contains 60,000 32×32 color images in 10 categories, with 50,000 images in the training set and 10,000 images in the test set; The CIFAR-100 dataset contains 60,000 32×32 color images across 100 categories, with 50,000 images in the training set and 10,000 images in the test set. SVHN dataset: A dataset containing street address numbers for digit recognition, consisting of approximately 600,000 images; Medical imaging subset: Selected multicenter lung CT image data, including 10,000+ image samples collected by equipment from different hospitals, labeled with lesion detection results; Financial Risk Dataset: Contains credit data from different regions, with a total of 50,000+ samples. Features include income, liabilities, credit history, etc., and the label is the default risk level.

[0062] (2) Experimental setup: The experimental environment consisted of an Ubuntu 22.04 operating system, equipped with an AMD EPYC 771364-core processor, 120GB of RAM, and an NVIDIA L40 GPU. The model was implemented using PyTorch 2.2.2 (CUDA 12.4) and DGL 2.2.0. Training parameters were set as follows: an initial learning rate of 1×10⁻², gradually decreasing to 1×10⁻². 4 The random dropout rate is 0.1; the maximum number of training rounds is 30; the client participation rate is 100%; and the data heterogeneity α = 0.2 (default setting).

[0063] (3) Experimental results and analysis: The experiment compares the performance of the Regularization-Aware Client Selection Federated Learning (RACS-FL) proposed in this invention with the benchmark algorithm in terms of three core dimensions: accuracy, convergence speed, and fairness. The results are shown in Tables 1 and 2 below: Table 1: Comparison of local accuracy on different datasets (number of clients = 30)

[0064] Table 2: Comparison of global accuracy on different datasets (number of clients = 30)

[0065] Experimental results show that: Accuracy Improvement: The RACS-FL proposed in this invention significantly outperforms the benchmark algorithm in both local and global accuracy on all datasets, with an average improvement of 8.5%. In particular, on multi-class, highly heterogeneous datasets such as CIFAR-100, the improvement exceeds 14%, verifying its adaptability to complex and heterogeneous scenarios. Convergence speed optimization: The number of convergence rounds required for the RACS-FL proposed in this invention to achieve stable performance is reduced by an average of 12.3%, and only 18 rounds are needed to converge on CIFAR-100, which is far lower than the 28 rounds of the Federated Averaging (FedAvg) algorithm, significantly improving training efficiency; Enhanced fairness: The fairness index (standard deviation of performance of each client) of the RACS-FL proposed in this invention is reduced by an average of 15.7%, indicating that it can effectively balance the performance differences of different clients and improve the overall robustness of the system; Real-world application: On real-world datasets such as medical imaging and financial risk, the RACS-FL proposed in this invention still maintains a performance improvement of 7.6% to 9.2%, verifying its practical application value.

[0066] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-described technical content to create equivalent embodiments without departing from the scope of the present invention. The implementation schemes in the above embodiments can also be further combined or replaced. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A federated learning method based on multi-objective client selection and Moro envelope regularization, characterized in that, include: S1. Data Preprocessing and Heterogeneous Scene Construction: Preprocess the private data stored in a decentralized manner on each client and build a federated learning heterogeneous data environment; S2, Multi-target client selection: For each round of federated training, evaluate the data quality score, model performance score, resource efficiency score, and system fairness score for each client; including: Data quality assessment: This involves quantifying data diversity through category distribution entropy analysis, evaluating data distribution characteristics using statistical features such as mean and variance analysis, verifying data integrity through missing value detection and outlier identification, and ultimately generating a data quality score. Specifically, this includes: Data diversity quantification: Calculate data diversity scores using category distribution entropy; Distribution similarity measurement: Evaluate the consistency of data distribution by comparing the statistical characteristics of client data and global data; Data integrity check: Calculate a data integrity score by counting the number of missing samples in the client data; Anomaly identification: By counting the number of anomaly samples in the client data, anomaly constraint scores are calculated. Data quality score fusion: The data diversity score, data distribution consistency score, data integrity score, and outlier sample constraint score are weighted and fused to obtain the client's data quality score; Model performance evaluation: Based on the classification accuracy, training loss and convergence stability of the client's historical training, a model performance score is constructed, and clients that can provide high-value updates to the global model are given priority. Resource efficiency assessment: Based on the client's computing power, network bandwidth, and energy status, a resource efficiency score is generated to balance training performance and system resource constraints; Fairness assessment: By monitoring client participation rate, contribution, and selection balance, a fairness score is calculated to ensure the long-term stable operation of the system; Multi-objective fusion and selection: A comprehensive score is obtained by weighted summation and fusion of four dimensions. The Pareto optimal client subset is selected from all clients using a non-dominated sorting algorithm and crowding distance calculation. S3. Construct and train a federated learning model that incorporates Moro envelope regularization; The selected client trains the local model using a loss function that includes a local Moro envelope regularization term and a global Moro envelope regularization term; wherein the local Moro envelope regularization term is used to constrain the deviation between the local model parameters and the global model parameters, and the global Moro envelope regularization term is used to constrain the stability of the global model parameter updates. The local model training employs a min-max alternating optimization strategy, updating model parameters by alternately executing a minimization update phase and a maximization update phase, specifically including: The server sends the current global model parameters to the selected client, and the client initializes the local model parameters with the global model parameters and samples the current training batch data from the local training data. The client inputs the current training batch data into the local model to obtain the reconstruction output, prediction output, sequence association matrix and prior association matrix. It then calculates the reconstruction loss based on the reconstruction output and the original input, the cross-entropy loss based on the prediction output and the true label, and the association difference loss based on the sequence association matrix and the prior association matrix. During the minimization update phase, the parameters corresponding to the sequence association matrix are fixed, and the parameters corresponding to the local model parameters and the prior association matrix are updated to minimize the overall training loss. During the maximization update phase, the parameters corresponding to the prior association matrix are fixed, and the parameters corresponding to the sequence association matrix are updated to increase the association difference between the sequence association matrix and the prior association matrix. According to the preset number of alternating training cycles, the minimum update phase and the maximum update phase are repeatedly executed until the local training rounds are reached or the local convergence condition is met, so as to obtain the local model parameters after the current client training. The client uploads the trained local model parameters, training loss, prediction results or soft labels to the server so that the server can perform similarity weighted aggregation and the next round of global model update; Each client uploads the trained local model parameters to the server; S4. Knowledge distillation and similarity-weighted aggregation: The server generates a global knowledge representation based on the local models uploaded by each client, calculates similarity weights based on the consistency between each local model and the global model, and uses the similarity weights to perform weighted aggregation of local model parameters to update the global model. The server receives the local model parameters, local prediction results, and local feature representation uploaded by the selected client in the current round, and generates a global knowledge representation based on the initial aggregation weights. Based on the initial aggregated weights, a global soft prediction result and a global feature representation are generated; The server calculates the consistency similarity between client i and the global knowledge representation; The server calculates the normalized similarity weight of client i based on the consistency similarity. The server performs weighted aggregation of the local model parameters uploaded by each client based on the normalized similarity weights to obtain intermediate global model parameters; The server modifies the intermediate global model parameters by combining the global Moro envelope regularization term to obtain the global model parameters for the (t+1)th round. The server will update the global model parameters in round t+1. It will be distributed to the next selected clients for subsequent local model training; S5. Iteration and Output: Repeat steps S2 through S4 until the preset convergence condition is met, and output the final global model and the personalized local model for each client.

2. The method according to claim 1, characterized in that, The preprocessing of private data stored separately by each client and the construction of a federated learning heterogeneous data environment include: The benchmark dataset is divided using the Dirichlet distribution. The degree of data heterogeneity is controlled by adjusting the distribution parameter α. For each data category, a K-dimensional allocation ratio vector is generated and the samples are allocated to K clients according to the generated ratio. This makes the data of each client exhibit unbalanced category distribution and different feature distribution characteristics, so as to construct a federated learning heterogeneous data environment.

3. The method according to claim 1, characterized in that, The method employs a non-dominated sorting algorithm and crowding distance calculation to select the optimal subset of Pareto clients from all clients, including: Clients are divided into different frontier layers using non-dominated sorting. Within the same frontier layer, the most representative client is selected based on crowding distance. The calculation is as follows: ； in, Let i be the value of client i on target j. Let m be the maximum and minimum values ​​for all clients on target j, and m be the number of targets.

4. The method according to claim 1, characterized in that, The multi-objective client selection also includes a step of multi-objective optimization using a multi-objective optimization function, specifically including: The multi-objective optimization function includes at least the knowledge distillation loss. Cross-entropy loss Feature alignment loss The loss due to Moro envelope regularization is expressed as: ； in, For client-side local model parameters, For distillation-related parameters, For hyperparameters, This represents the local Moro envelope regularization term. This represents the global Moro envelope regularization term.

5. The method according to claim 1, characterized in that, The comprehensive training loss is expressed as: ； in, This represents the overall training loss during the minimization phase. Represents cross-entropy loss, This represents the knowledge distillation loss. Indicates feature alignment loss, This represents the local Moro envelope regularization term. This represents the global Moro envelope regularization term. Indicates the reconstruction loss. Indicates related difference loss, These represent the weighting coefficients of the corresponding loss terms; The optimization objective in the maximization phase is expressed as: ； in, This represents the comprehensive training loss during the maximization phase. By introducing a negative correlation difference term into the loss function, the model increases the difference between the sequence correlation matrix and the prior correlation matrix when updating the sequence correlation parameters.

6. The method according to claim 1, characterized in that, The local model parameters are weighted and aggregated using the similarity weights to update the global model, including: The local model parameters uploaded by client i are denoted as The local prediction result is denoted as Local features are denoted as ; The initial aggregation weights are expressed as follows: ； in, This represents the initial aggregate weight of client i. This represents the number of local samples for client i. This represents the set of clients selected in round t.

Images