Personalized federated learning method and system for non-independent and identically distributed data

Abstract

The application relates to a personalized federated learning method and system for non-independent and identically distributed data, and belongs to the technical field of federated learning. The method solves the problems of insufficient personalized adaptation and high privacy leakage risk in the existing non-independent and identically distributed data scene. The method comprises the following steps: a server sends global model parameters to each client; each client trains a local main model based on local data and the received global model parameters, obtains a local parameter increment, and uploads the local parameter increment to the server; time sequence data features are extracted from the local data, and the time sequence data features are transmitted into a trained personalized modulator together with task preference labels to generate local adaptation parameters; the local adaptation parameters are used to adjust the global model parameters to generate a local personalized model for local reasoning; the server receives the local parameter increments of each client, aggregates the local parameter increments to update the global model parameters, and enters the next round of iteration until the iteration end condition is met. The precision of the personalized model and the privacy protection degree are improved.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and in particular to a personalized federated learning method and system for non-independent and identically distributed data. Background Technology

[0002] With the rapid development of big data and artificial intelligence technologies, federated learning has emerged as a new distributed machine learning paradigm in scenarios involving cross-institutional collaboration and application of machine learning models. Federated learning allows participating parties (called clients) to collaboratively train a global machine learning model without sharing their local raw data, solely by exchanging model parameters or gradient updates. This effectively balances the conflict between data value extraction and privacy protection. This technology has already demonstrated enormous application potential in fields such as healthcare and smart cities.

[0003] However, traditional federated learning methods (such as the FedAvg algorithm) rely on the ideal assumption that all client data follows an independent and identically distributed (IID) distribution. In real-world applications, especially in scenarios such as medical image diagnosis (where patient groups and disease profiles vary across hospitals), personalized recommendations (where different users have different interests and preferences), and smart agriculture (where climate and soil conditions differ across regions), client data naturally exhibits high heterogeneity, meaning the data follows a non-independent and identically distributed (Non-IID) distribution. To address the performance degradation caused by Non-IID data and the need for client personalization, existing technologies typically employ global model fine-tuning strategies, regularization-based personalization methods, and hypernetwork-based model generation.

[0004] Existing technologies face the following technical problems and shortcomings when addressing the challenges of Non-IID data in federated learning: the global model is difficult to adapt to the local data distribution of each client, resulting in a significant reduction in model performance on specific clients; purely localized training loses the advantages of collaboration, while forcing a unified model cannot meet personalized needs, creating a dilemma; existing personalization methods often require uploading some personalized parameters, posing a risk of privacy leakage; and they fail to fully utilize the long-term evolution of client data distribution, making them unable to adapt to dynamically changing data environments. Summary of the Invention

[0005] Based on the above analysis, the embodiments of the present invention aim to provide a personalized federated learning method and system for non-independent and identically distributed data, in order to solve the problems of insufficient personalized adaptation, static and rigid models, and high privacy leakage risks caused by the lack of modeling ability for the dynamic evolution of client data in existing non-independent and identically distributed data scenarios.

[0006] On one hand, embodiments of the present invention provide a personalized federated learning method for non-independent and identically distributed data, which includes the following steps in the federated learning iteration process performed between the server and multiple clients:

[0007] The server sends global model parameters to each client;

[0008] Each client performs local processing, including: training a local master model based on local data and received global model parameters, obtaining local parameter increments and uploading them to the server; extracting time-series data features from local data and passing them along with task preference labels into a trained personalized modulator to generate local adaptation parameters; and adjusting global model parameters using local adaptation parameters to generate a local personalized model for local inference.

[0009] The server receives local parameter increments from each client, aggregates them to update global model parameters, and enters the next iteration until the iteration termination condition is met.

[0010] Based on further improvements to the above method, the personalized modulator is a localized neural network model built on a recurrent neural network. The dimension of its output local adaptation parameters is determined by the number and type of adjustable parameter layers issued by the server, or by the client automatically parsing the global model parameters.

[0011] Based on further improvements to the above method, the local adaptation parameters include: a first parameter that scales and adjusts the weights of the corresponding layer in the global model parameters, and / or a second parameter that offsets and adjusts the biases of the corresponding layer in the global model parameters.

[0012] Based on the above method, further improvements are made, including the following steps before performing federated learning iterations:

[0013] Each client establishes its local master model and initializes its personalized modulator based on the initial global model parameters sent by the server.

[0014] Temporal data features and task preference labels are extracted from local data, and the personalized modulator is pre-trained in an end-to-end joint training manner. After training, the parameters of the trained personalized modulator are retained, and the parameters of the local master model are restored to the initial global model parameters issued by the server.

[0015] Further improvements to the above methods include end-to-end joint training approaches, such as:

[0016] The personalized modulator to be trained is connected to the local master model. The local adaptation parameters output by the personalized modulator are used to adjust the parameters of the local master model to obtain personalized parameters and form a local temporary model. The local temporary model is used to perform task inference using local data, and the loss is calculated based on the loss function. The parameters of the personalized modulator are updated based on the loss through backpropagation.

[0017] Based on the further improvement of the above method, the loss function is a composite loss function, including: task loss term, deviation penalty term, and regularization term; wherein, the task loss term is calculated based on the prediction results of the local temporary model; the deviation penalty term is obtained by calculating the difference between the adjusted personalized parameters and the initial global model parameters; and the regularization term is used to constrain the adaptation parameters themselves.

[0018] Based on further improvements to the above method, during the federated learning iteration process, each client determines whether to perform local incremental training and update the personalized modulator according to preset triggering conditions.

[0019] Based on further improvements to the above method, the triggering conditions include at least one of the following: the performance of the local personalized model on the local validation set meets a preset decline condition, a drift in the local data distribution is detected, and a preset update cycle is reached.

[0020] Based on the further improvement of the above method, the weight of the deviation penalty term dynamically decays as the number of training iterations increases.

[0021] On the other hand, embodiments of the present invention provide a personalized federated learning system for non-independent and identically distributed data, comprising: a server and at least one client; wherein,

[0022] The server includes: a global model management module for storing, distributing, and updating global model parameters; a secure aggregation module for receiving and aggregating local parameter increments from each client; and a secure communication module for distributing global model parameters from the global management module to each client, and receiving local parameter increments from each client and forwarding them to the global model management module.

[0023] The client includes: a local master model module, used to train a local master model based on local data and received global model parameters to obtain local parameter increments; a personalized modulator module, used to generate local adaptation parameters by inputting time-series data features extracted from local data along with task preference labels into a pre-trained personalized modulator; a personalized model generation module, used to adjust global model parameters using local adaptation parameters to generate a local personalized model; and a secure communication submodule, used to receive global model parameters from the secure communication module and upload local parameter increments to the secure communication module.

[0024] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0025] 1. Through a local personalized modulator on the client, the system actively learns and memorizes the temporal evolution patterns of local data distribution. This modulator can dynamically generate refined adaptation parameters based on historical and current data characteristics, and "fine-tune" the global model. This results in a local personalized model that deeply matches the client's unique and time-varying data characteristics, thereby improving model accuracy and user satisfaction in highly non-IID scenarios such as medical diagnosis and recommendation systems.

[0026] 2. The training data, model parameters, training process, and generated local adaptation parameters of the personalized modulator are all in the client's local environment and are not transmitted to the server in any form; the server only receives and aggregates parameter increments used to update the global model, which enhances privacy protection.

[0027] 3. A binary decoupled architecture of "global master model" and "local personalized modulator" is adopted. The global master model learns general knowledge and feature representations across clients through federated aggregation, maintaining the model's collaboration and generalization. The local personalized modulator focuses on learning the mapping strategy to accurately adapt the general model to the local time-varying data distribution. The two perform their respective functions and complement each other, fundamentally solving the fundamental contradiction between "personalization and collaboration".

[0028] 4. The personalized modulator initiates fully local online training during the intervals of federated learning iterations based on triggering conditions, enabling personalized strategies to keep pace with the dynamic changes in the client's data environment and achieve adaptive optimization throughout the entire lifecycle, effectively addressing the challenge of "static model being out of sync with dynamic environment".

[0029] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0030] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0031] Figure 1 This is a flowchart of the personalized federated learning method for non-independent and identically distributed data in Embodiment 1 of the present invention;

[0032] Figure 2This is a schematic diagram of the structure of the personalized federated learning system for non-independent and identically distributed data in Embodiment 2 of the present invention. Detailed Implementation

[0033] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0034] Example 1

[0035] A specific embodiment of the present invention discloses a personalized federated learning method for non-independent and identically distributed data. During the federated learning iteration process executed between the server and multiple clients, such as... Figure 1 As shown, it includes the following steps:

[0036] S1. The server sends global model parameters to each client.

[0037] S2. Each client performs local processing, including: training the local master model based on local data and received global model parameters, obtaining local parameter increments and uploading them to the server; extracting time-series data features from local data and passing them along with task preference labels into the trained personalized modulator to generate local adaptation parameters; and adjusting the global model parameters using the local adaptation parameters to generate a local personalized model for local inference.

[0038] S3. The server receives the local parameter increments from each client, aggregates them to update the global model parameters, and enters the next iteration until the iteration termination condition is met.

[0039] It should be noted that this embodiment uses "federated learning for multi-center medical image diagnosis" as a typical application scenario, taking the collaborative training of a CT image tumor classification model by three hospitals (clients) as an example. Each hospital has a different patient population and disease distribution, representing non-independent and identically distributed data. It should be understood that the application scenario described herein is only for explaining the embodiments of the present invention and is not intended to limit it.

[0040] The personalized modulator trained in step S2 is obtained by pre-training before iteratively executing federated learning. Specifically, before executing the federated learning iterations, an initialization phase is also included, which is executed only once and includes the following steps:

[0041] ① Each client establishes its local master model and initializes its personalized modulator based on the initial global model parameters issued by the server.

[0042] The server initializes a global master model (e.g., a ResNet-50 model for binary classification of "benign / malignant") and generates initial global model parameters. It is sent to each client through a secure communication channel.

[0043] Preferably, the server defines and stores a list of adjustable parameter layers, specifying which parts of the model can be customized by the client (e.g., specifying that the parameters of all convolutional and fully connected layers can be customized). This list is distributed along with the global model parameters. This approach allows the server to have global control over the granularity of customization, preventing clients from modifying key underlying structures and ensuring a solid foundation for federated collaboration.

[0044] Each client (three hospitals) downloads the initial global model parameters from the server through a secure communication channel, establishes a local master model, and the network structure and parameters are completely consistent with the global model parameters, serving as the carrier for subsequent local training and receiving global updates; and initializes the personalized modulator.

[0045] It should be noted that the personalized modulator is a localized neural network model built on a recurrent neural network. The personalized modulator is used to learn the mapping relationship from the temporal data features of the local data distribution to the effective adaptation parameters. It receives local temporal data features and task preference labels, and outputs a set of local adaptation parameters.

[0046] The local adaptation parameters output by the personalized modulator include: a first parameter that scales and adjusts the weights of the corresponding layer in the global model parameters, and / or a second parameter that offsets and adjusts the biases of the corresponding layer in the global model parameters. The weights and biases are parameter types for each layer, and whether or not the biases of certain layers need adjustment is determined based on actual requirements or through a configuration file.

[0047] If the server issues a list of adjustable parameter layers, the dimension of the local adaptation parameters output by the personalized modulator is determined by the number and type of adjustable parameter layers.

[0048] For example, if the list of adjustable parameter layers specifies L adjustable parameter layers, and each layer needs to generate adaptation parameters to adjust the weights and biases, then the output dimension of the personalized modulator is automatically set to 2L. This automatic adaptation mechanism greatly enhances compatibility with different model architectures and ease of deployment.

[0049] If the server does not distribute a list of adjustable parameter layers, the client automatically parses the global model parameters and determines the number of layers that need to be adapted and adjusted based on its own needs, thereby determining the output dimension of the personalized modulator.

[0050] In this embodiment, the personalized modulator improves the recurrent neural network by using a bidirectional LSTM and introducing an attention mechanism, enabling the model to focus on the part of the historical sequence that is most relevant to the current data distribution.

[0051] Specifically, the personalized modulator consists of, in sequence: an input layer, a bidirectional LSTM layer, an attention layer, and an output layer; among which,

[0052] The input layer is used to organize multivariate time series by time step, with each time step corresponding to a multidimensional feature vector, and constructs a time series feature tensor to be passed to the bidirectional LSTM layer;

[0053] In the bidirectional LSTM layer, the forward LSTM processes in forward chronological order to capture the historical context, while the backward LSTM processes in reverse chronological order to capture the future context. The hidden states from both directions are then concatenated and passed to the attention layer.

[0054] The attention layer calculates the relevance score between each time step and the query vector, obtains the attention weights by Softmax normalization, adaptively weights and sums the outputs of the bidirectional LSTM at different time steps, and passes the output context vector to the output layer.

[0055] The output layer generates the original output by linearly transforming the context vector through the fully connected layer, and then maps it to local adaptation parameters through a custom activation function: the first parameter in the original output is applied with the Sigmoid function and scaled to [0.8, 1.2]; the second parameter in the original output is applied with the Tanh function and scaled to [-0.1, 0.1].

[0056] ② Extract time-series data features and task preference labels from local data, pre-train the personalized modulator in an end-to-end joint training manner, retain the parameters of the trained personalized modulator after training, and restore the parameters of the local main model to the initial global model parameters issued by the server.

[0057] It should be noted that this step is performed independently on the client side. First, preprocessing operations are performed on the locally stored data, including but not limited to: data cleaning and normalization.

[0058] Furthermore, extracting time-series data features and task preference labels from local data involves extracting data distribution features from the preprocessed local data according to the time series, and determining task preference labels based on the task objectives. Data distribution features are used to capture the variability and trend of data distribution over time, such as statistical features (mean / median, variance / standard deviation, peak value), trend features (linear regression slope), volatility features (rolling variance / standard deviation, range), and distribution shape features (histogram). Task preference labels are used to indicate the client's emphasis on specific category performance in the model's prediction results, such as recall and precision.

[0059] For example, Hospital A, one of the three hospitals, is a cancer specialty hospital that prefers high recall. Based on its own clinical needs, it sets a recall weight of 0.8 (indicating extreme sensitivity to "missed diagnoses" of malignant tumors and a tendency to classify suspicious cases as malignant). Then, from its historical medical record database, it calculates the "detection rate of malignant tumors in CT images" for the past 12 months, forming a time-series data feature of length 12 [0.68, 0.69, ..., 0.75]. Simultaneously, each time point is associated with Hospital A's task preference label [recall weight = 0.8].

[0060] It should be noted that in a binary classification scenario, the input time-series data features can be simplified to a data sequence of the proportion of malignant tumors; the task preference labels can be simplified to recall weights for the malignant tumor category. In a multi-class classification scenario, each time point can correspond to the proportion features of multiple categories and multiple task preference labels.

[0061] Furthermore, an end-to-end joint training approach is adopted, including:

[0062] The personalized modulator to be trained is connected to the local master model. The local adaptation parameters output by the personalized modulator are used to adjust the parameters of the local master model to obtain personalized parameters and form a local temporary model. The local temporary model is used to perform task inference using local data, and the loss is calculated based on the loss function. The parameters of the personalized modulator are updated based on the loss through backpropagation.

[0063] Specifically, the extracted time-series data features and task preference labels are formed into feature samples according to a preset fixed length, a feature sample set is constructed, and input into the personalized modulator. A set of local adaptation parameters is output, which are immediately applied to the corresponding layer of the local main model to form a local temporary model.

[0064] The parameters of the adjustable parameter layer of the local master model are adjusted using the following formula:

[0065] ,

[0066] in, and These represent the local temporary model number. The weights and biases adjusted by an adjustable parameter layer. and These represent the parameters of the current global model (i.e., the local master model) at the 1st, 2nd, and 3rd positions, respectively. The weights and biases of each adjustable parameter layer before adjustment. and These respectively represent the local adaptation parameters and the first one. The first and second parameters corresponding to each adjustable parameter layer.

[0067] Furthermore, a data sample set is constructed based on real local data (such as stored CT image data) and labeled categories; data samples are extracted from the data sample set and input into the local temporary model for task inference to obtain prediction results.

[0068] Then, the loss is calculated based on the loss function. The loss function is a composite function, including: task loss term. Deviation penalty items and regularization term The formula is shown below:

[0069] ,

[0070] in, The weights representing deviations from the penalty term increase with the number of training iterations. The dynamic decay occurs as the constraint increases; strong constraints are applied initially to ensure stability, while relaxation is applied later to pursue better personalized effects. This indicates the preset initial weight, such as 0.1; This represents the weight of the regularization term, set to a small fixed value, such as 0.001; This represents an exponential function.

[0071] Specifically, task loss items It is calculated based on the prediction results of the local temporary model, with the goal of making the locally personalized model more accurate on local data. A weighted cross-entropy loss function is typically used.

[0072] For example, for hospital A, the weight of the malignancy category is determined based on the task preference label, which directly guides the modulator to learn and generate adaptation parameters that can improve the recall rate, as shown in the following formula:

[0073] ,

[0074] in, Indicates the first The weights of each category, Indicates the total number of categories. Indicates the first One-hot encoding for each category, Indicates the first prediction made by the local master model. The probability of each category.

[0075] Deviation penalty item This is obtained by calculating the difference between the adjusted personalized parameters and the initial global model parameters. This term prevents the personalized modulator from making overly aggressive adjustments in the early stages of training, avoiding a significant deviation between the local personalized model and the global consensus, thus maintaining the cornerstone of federated collaboration. Specifically, this includes a deviation penalty term. The formula is shown below:

[0076] ,

[0077] in, and These represent the adjusted personalized parameters and the initial global model parameters, respectively. This indicates the calculation of the L2 norm.

[0078] Regularization term Used to constrain the adaptation parameters themselves, so that the first parameter The second parameter should be as close to 1 as possible. Keep it as close to 0 as possible. This ensures that the adaptation parameters of the personalized modulator output are reasonable, avoiding extreme values ​​that could lead to model instability or overfitting.

[0079] Specifically, regularization terms The formula is shown below:

[0080] .

[0081] Through backpropagation, the loss gradient is used only to update the parameters of the personalized modulator. The Adam optimizer is used during training, with an initial learning rate of 0.001, decaying to 0.9 every 10 epochs. A crucial step is that after each training epoch, the parameters of the local master model are reset back to the initial global model parameters. This allows the RNN modulator to learn a general, decoupled "adaptation strategy" from specific model parameters, learning how to adjust any given model when the local data distribution varies and specific preferences are set.

[0082] Through the execution of the initial phase described above, each client obtains a trained personalized modulator. For example, Hospital A receives a personalized modulator that has localized its data evolution trend (gradual increase in malignancy rate) and clinical needs (high recall rate). Throughout the process, historical medical record data, modulator parameters, and training gradients are all stored within the hospital, achieving zero-privacy initialization of personalized capabilities.

[0083] Steps S1-S3 constitute the federated learning iterative process. After the server distributes the global model parameters to each client in step S1, each client performs two tasks in parallel in step S2: S21, generating local parameter increments that contribute to the global model and uploading them to the server; and S22, generating a personalized model for local use. Step S3 is crucial for federated learning to achieve knowledge collaboration, completing global aggregation on the server side and entering the next iteration.

[0084] Specifically, in step S21, each client first preprocesses the newly collected data and puts it into a data sample set, from which data samples for the current round are selected.

[0085] It should be noted that if the number of new data samples is greater than the minimum number required to train the local master model, they can be directly selected from the new data. Otherwise, data is retrieved from the new data samples based on a fixed time window to obtain the data samples for the local master model in the current round, ensuring that the selected data samples each time contain the latest data and have a sufficient number of samples to support effective gradient descent.

[0086] Furthermore, the client updates its local master model based on the received global model parameters, and performs multiple rounds of local training on the local master model using the data samples from the current round, learning common knowledge that can be contributed to the global model. After training, the local parameter increment is calculated.

[0087] Meanwhile, in step S22, new data distribution features are extracted from the newly collected data and placed into the feature sample set. A fixed length of data distribution features (e.g., 5, i.e., retaining the data distribution features of the last 5 rounds) is obtained from the new data distribution features and used as the time-series data features of the personalized modulator in the current round. This ensures that the time-series data features include both the latest data distribution features and the historical multi-round data distribution features accumulated during the federated learning iteration process.

[0088] After adding task preference labels to each time step of the time series data features in the current round, these labels are fed into the trained personalized modulator to analyze the evolution of the sequence (e.g., the proportion of malignant data increases slightly over multiple rounds). Combined with task preferences, a new set of more refined local adaptation parameters is inferred and output, forming the fine-tuning instructions for the pair.

[0089] The received global model parameters are adjusted based on the local adaptation parameters to generate a local personalized model. This personalized model is only used for the client's local inference tasks, does not participate in federated aggregation, and its parameters do not leave the client.

[0090] Once the client receives the local parameter increment, it can simultaneously encrypt it and upload it to the server via a secure communication channel. During this process, the parameters of the personalized modulator, its generated adaptation parameters, and the local personalized model parameters are strictly kept locally and never disclosed, eliminating any possibility of privacy leaks during the personalization process.

[0091] It should be noted that during the federated learning iteration process, such as after extracting new data distribution features in step S22, or while waiting for the next iteration, or when the client is idle, each client determines whether to perform local incremental training on the personalized modulator and update it according to the preset trigger conditions, so as to adapt it to the changes in the new data distribution.

[0092] The triggering conditions include at least one of the following: the performance of the local personalized model on the local validation set meets the preset decline condition, a drift in the local data distribution is detected, and a preset update cycle is reached.

[0093] It should be noted that the local validation set is a fixed, high-quality validation set. For example, a fixed dataset of 1000 precisely labeled CT images is maintained within Hospital A. This dataset covers various typical and complex cases and is not used for any training. The performance of the local personalized model is evaluated using the local validation set, and metrics such as recall and precision are recorded. Performance is considered to have "decreased" when it meets any of the preset decline conditions, including: a decrease in recall exceeding a certain threshold compared to the highest historical recall; or a continuous decline in recall exceeding a certain threshold over multiple rounds.

[0094] It should be noted that during each iteration, the core distribution characteristics of the data in each round (such as the proportion of malignant cases) are recorded. The Page-Hinkley Test (PHT) algorithm is used to analyze each core distribution characteristic. If the detection statistic of any core distribution characteristic exceeds its corresponding threshold, it is considered that "data distribution has drifted".

[0095] Reaching a preset update cycle is a periodic triggering mechanism, such as being automatically triggered after every N rounds (e.g., 20 rounds) of federated learning iterations.

[0096] Once any of the triggering conditions are met, the client will use the data distribution characteristics of the recent period to perform a small number of incremental training rounds on the personalized modulator locally. During training, it will still be connected to the local master model and adopt an end-to-end joint training method. After incremental training, the parameters of the local master model will be restored to the state before the training started (the global model parameters downloaded from the server at that time) to ensure that the subsequent federated learning process will not be affected by this temporary adjustment.

[0097] Preferably, during incremental training, elastic weight consolidation (EWC) technology is introduced to prevent the personalized modulator from forgetting the old patterns it has learned while adapting to new changes.

[0098] It should be noted that in step S3, the server receives and decrypts the encrypted local parameter increments from all clients via a secure communication module. A predetermined aggregation algorithm (such as federated averaging or federated momentum) is used to aggregate the decrypted parameter increments, yielding an aggregation result. The server uses this aggregation result to update the global model parameters, which incorporate the common knowledge contributed by each client. If the iteration termination condition is not met (e.g., reaching the maximum number of iterations or global model convergence), the next iteration begins. The global model continuously evolves through collaboration, while each client's personalized modulator continuously self-optimizes as the data evolves. If the iteration termination condition is met, the iteration ends, resulting in an optimized global model and highly personalized local models for each client.

[0099] Compared with existing technologies, the personalized federated learning method for non-independent identically distributed data provided in this embodiment actively learns and memorizes the temporal evolution of local data distribution through a local personalized modulator on the client. This modulator can dynamically generate refined adaptation parameters based on historical and current data characteristics, "fine-tuning" the distributed global model. This ensures that the final generated local personalized model deeply matches the client's unique and time-varying data characteristics, thereby improving model accuracy and user satisfaction in highly non-IID scenarios such as medical diagnosis and recommendation systems. The training data, model parameters, training process, and generated local adaptation parameters of the personalized modulator all reside in the client's local environment and are not transmitted to the server in any form. The server only receives and aggregates the data. The system enhances privacy protection by updating the parameters of the global model incrementally. It adopts a binary decoupled architecture of "global main model" and "local personalized modulator". The global main model learns general knowledge and feature representations across clients through federated aggregation, maintaining the model's collaboration and generalization. The local personalized modulator focuses on learning the mapping strategy to accurately adapt the general model to the local time-varying data distribution. The two perform their respective functions and complement each other, fundamentally solving the fundamental contradiction between "personalization and collaboration". The personalized modulator starts fully local online training between federated learning iterations based on trigger conditions, so that the personalized strategy can keep up with the dynamic changes of the client data environment and achieve adaptive optimization throughout the life cycle, effectively addressing the challenge of "static model and dynamic environment decoupling".

[0100] Example 2

[0101] Another embodiment of the present invention discloses a personalized federated learning system for non-independent and identically distributed data, thereby implementing the personalized federated learning method for non-independent and identically distributed data in Embodiment 1. The specific implementation of each module is described in the corresponding description in Embodiment 1. The system includes: a server and at least one client; wherein,

[0102] The servers include:

[0103] The global model management module is used to store, distribute, and update global model parameters;

[0104] The security aggregation module is used to receive local parameter increments from each client and aggregate them.

[0105] The secure communication module is used to send the global model parameters of the global management module to each client, and to receive the local parameter increments of each client and forward them to the global model management module.

[0106] The client includes:

[0107] The local master model module is used to train the local master model based on local data and received global model parameters, and obtain local parameter increments.

[0108] The personalized modulator module is used to generate local adaptation parameters by inputting time-series data features extracted from local data, along with task preference labels, into the trained personalized modulator.

[0109] The personalized model generation module is used to adjust the global model parameters using local adaptation parameters to generate a local personalized model.

[0110] The secure communication submodule is used to receive global model parameters issued by the secure communication module, and to upload local parameters incrementally to the secure communication module.

[0111] It should be noted that the server's secure communication module is responsible for establishing and maintaining encrypted communication links (such as TLS / SSL) with all clients. The client's secure communication submodule works in pairs with the server's secure communication module. The two modules work together to ensure the confidentiality and integrity of all interactions between the client and the server.

[0112] Furthermore, the client also includes:

[0113] The local data management module is used to collect and store local data, preprocess the local data, and extract data distribution features.

[0114] The personalized modulator training module is used to perform initial and incremental training of the personalized modulator based on the data distribution characteristics in the local data management module.

[0115] Local Inference Service Module: Used to input the data to be inferred into the local personalized model in the personalized model generation module to obtain the task inference result.

[0116] Based on the method in Example 1, the schematic diagram of the collaborative interaction between the modules in this system is as follows: Figure 2 As shown. Since this embodiment of a personalized federated learning system for non-independent and identically distributed data can draw upon the aforementioned personalized federated learning method for non-independent and identically distributed data, and this is a repetition, it will not be repeated here. Because this system embodiment shares the same principle as the aforementioned method embodiment, it also possesses the corresponding technical effects of the aforementioned method embodiment.

[0117] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0118] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A personalized federated learning method for non-independent and identically distributed data, characterized in that, The federated learning iteration process, performed between the server and multiple clients, includes the following steps: The server sends global model parameters to each client; Each client performs local processing, including: training a local master model based on local data and the received global model parameters, obtaining local parameter increments and uploading them to the server; extracting time-series data features from the local data, and inputting them along with task preference labels into a trained personalized modulator to generate local adaptation parameters; and adjusting the global model parameters using the local adaptation parameters to generate a local personalized model for local inference. The server receives the local parameter increments from each client, aggregates them to update the global model parameters, and enters the next iteration until the iteration end condition is met. Before executing the federated learning iteration, the process includes: each client establishing a local master model based on the initial global model parameters issued by the server, and initializing a personalized modulator; extracting time-series data features and task preference labels from local data, pre-training the personalized modulator in an end-to-end joint training manner, and retaining the parameters of the trained personalized modulator after training, while restoring the parameters of the local master model to the initial global model parameters issued by the server; the end-to-end joint training method includes: connecting the personalized modulator to be trained with the local master model, applying the local adaptation parameters output by the personalized modulator to adjust the parameters of the local master model to obtain personalized parameters and form a local temporary model; using local data to perform task inference on the local temporary model and calculating the loss according to the loss function; and updating the parameters of the personalized modulator based on the loss through backpropagation.

2. The personalized federated learning method for non-independent and identically distributed data according to claim 1, characterized in that, The personalized modulator is a localized neural network model built on a recurrent neural network. The dimension of its output local adaptation parameters is determined by the number and type of adjustable parameter layers issued by the server, or by the client automatically parsing the global model parameters.

3. The personalized federated learning method for non-independent and identically distributed data according to claim 1 or 2, characterized in that, The local adaptation parameters include: a first parameter for scaling the weights of the corresponding layer in the global model parameters, and / or a second parameter for offsetting the biases of the corresponding layer in the global model parameters.

4. The personalized federated learning method for non-independent and identically distributed data according to claim 1, characterized in that, The loss function is a composite loss function, including: a task loss term, a deviation penalty term, and a regularization term; wherein, the task loss term is calculated based on the prediction results of the local temporary model; the deviation penalty term is obtained by calculating the difference between the adjusted personalized parameters and the initial global model parameters; and the regularization term is used to constrain the adaptation parameters themselves.

5. The personalized federated learning method for non-independent and identically distributed data according to claim 1, characterized in that, During the federated learning iteration process, each client determines whether to perform local incremental training and update the personalized modulator based on preset triggering conditions.

6. The personalized federated learning method for non-independent and identically distributed data according to claim 5, characterized in that, The triggering conditions include at least one of the following: the performance of the local personalized model on the local validation set meets a preset decline condition, a drift in the local data distribution is detected, and a preset update cycle is reached.

7. The personalized federated learning method for non-independent and identically distributed data according to claim 4, characterized in that, The weight of the deviation penalty term decreases dynamically as the number of training iterations increases.

8. A personalized federated learning system for non-independent and identically distributed data, used to implement the personalized federated learning method for non-independent and identically distributed data as described in claim 1, characterized in that, include: A server and at least one client; wherein, The server includes: a global model management module for storing, distributing, and updating global model parameters; a secure aggregation module for receiving and aggregating local parameter increments from each client; and a secure communication module for distributing the global model parameters from the global management module to each client, and receiving and forwarding local parameter increments from each client to the global model management module. The client includes: a local master model module, used to train a local master model based on local data and the received global model parameters to obtain local parameter increments; a personalized modulator module, used to generate local adaptation parameters by inputting time-series data features extracted from local data, along with task preference labels, into a trained personalized modulator; a personalized model generation module, used to adjust the global model parameters using the local adaptation parameters to generate a local personalized model; and a secure communication submodule, used to receive global model parameters sent by the secure communication module and upload local parameter increments to the secure communication module.

Images