Federal learning model optimization method and system based on meta learning and regularization

Abstract

The invention discloses a federated learning model optimization method and system based on meta-learning and regularization, and belongs to the technical field of artificial intelligence and data privacy. Aiming at the problems of slow model convergence and insufficient individuation caused by non-independent identically distributed data, the method adopts collaborative optimization of a meta-learning double-circulation mechanism and regularization: a central server distributes a global model to a client, and the client performs internal circulation gradient updating and external circulation meta-gradient calculation by dividing a support set and a query set; generating updating information adapted to local data; meanwhile, an L2 regular term is introduced to constrain the parameter difference between a local model and a global model, and a personalized model is optimized in combination with prediction loss. The system realizes cooperative training through a global distribution module, a local adaptation module, a personalized module and an aggregation module, and the server adopts data volume weighted aggregation client updating. According to the method, the model generalization ability under heterogeneous data is effectively improved, and the prediction difference between the clients is balanced.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, machine learning, and data privacy protection, and specifically to a federated learning model optimization method and system based on meta-learning and regularization. Background Art

[0002] With the continuous advancement of artificial intelligence technology, big data-driven industrial transformation has profoundly impacted our daily lives. Large-scale modeling requires extensive training, while the traditional approach involves centralized learning of all data on a single device. However, with the continuous development of the mobile internet and growing awareness of personal privacy, traditional processing methods have become limited. Therefore, distributed modeling of neural networks is a key research topic in the field of federated learning.

[0003] A key prerequisite for federated learning is that data from each source is independent and identically distributed (IID). However, in practice, the non-IID nature of data across data sources makes traditional federated learning approaches ineffective. The heterogeneity of multi-source data leads to reduced model training efficiency and local model fairness, issues that personalized fair federated learning (PFL) aims to address.

[0004] Data heterogeneity makes it impossible for a single global model to meet the needs of all clients, and the accuracy of local models varies greatly. Existing algorithms such as FedAvg converge slowly and have weak generalization capabilities in non-IID scenarios. Although FedProx introduces regularization terms, it lacks the active optimization capabilities of the global model. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: in the federated learning scenario, to address the problems of decreased local model accuracy and insufficient personalized fairness caused by non-independent and identically distributed (non-IID) data, a collaborative modeling method is provided that combines meta-learning-driven global model optimization with regularization-constrained local updates. Through a double-loop mechanism, balanced control of model performance improvement under heterogeneous data and prediction differences between clients is achieved.

[0006] In order to solve the above problems, the present invention adopts the following technical means:

[0007] The present invention provides a federated learning model optimization method based on meta-learning and regularization, comprising the following steps:

[0008] Step 1: Global model distribution: The central server sends the global neural network parameters of the current iteration to the clients participating in the training. The global model parameters are generated by aggregating the update information uploaded by multiple clients in the previous iteration;

[0009] Step 2: Local adaptation of the global model: After receiving the global model parameters, the client performs two-stage training on the local data based on the meta-learning framework, including:

[0010] Inner loop update: divide the support set and query set from the local data, and use the support set to perform gradient update on the global model parameters to generate temporary parameters;

[0011] Outer loop optimization: Calculates the meta-gradient of temporary parameters based on the query set, adjusts global model parameters according to the preset learning rate, and generates client-specific update information;

[0012] Step 3: Local model personalization: Based on the global model parameters, the client optimizes the local model using regularization techniques, including:

[0013] Calculate the L2 norm difference between the local model parameters and the global model parameters to generate the proximal term;

[0014] Combining the proximal term with the prediction loss of the local model to form an overall loss function;

[0015] Performing gradient updates on local model parameters based on the total loss function to generate a personalized local model;

[0016] Step 4: Client model aggregation: The central server receives the update information uploaded by each client, performs weighted mean aggregation on the update information according to a preset weight calculation formula, and generates the global model parameters for the next round of iteration.

[0017] In the above scheme, the two-stage meta-learning technology includes:

[0018] Support set partitioning: Client k samples tasks T from the local training set i , divide the support set based on task T and queryset

[0019] Inner loop parameter update: by formula:

[0020]

[0021] The learning rate η on the support set is based on the client's requirements g With the meta-training step size k inner Run gradient descent to generate temporary global model parameters;

[0022] where θ i+1 and θ i Respectively represent the temporary global model parameters corresponding to the local model when the step size is i and i+1 in the meta-training phase, Represents in task T i The global model θ on the support set i The training loss is represents θ i gradient;

[0023] Outer loop gradient calculation: by formula:

[0024]

[0025] Perform new global model update gradient calculation on the query set based on the temporary global model obtained in the meta-training phase;

[0026] in and represent the global model before and after updating, β represents the learning rate in the meta-test phase, represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set, represents the loss function, Represents the global model parameters gradient.

[0027] In the above scheme, in the regularization technique of step 3:

[0028] The total loss function is:

[0029]

[0030] Where μ is the regularization strength, w t and Represent the local model and the global model respectively, Represents the actual loss value returned. The prediction error generated when the model makes prediction losses;

[0031] The gradient update formula is:

[0032]

[0033] where v k ′ and v k Represent the local personalized models before and after the update, Represents the gradient update information generated when the model predicts the loss, η l represents the local learning rate.

[0034] In the above scheme, in the weighted mean aggregation in step 4:

[0035] The weight calculation formula is:

[0036]

[0037] where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, D i represents the data volume of the i-th client, S t Represents the client number selected for the tth round of update;

[0038] The global model parameter update formula is:

[0039]

[0040] in, is the update amount of the global model by the i-th client in the t-th iteration, and They represent the global model parameters stored by the central server in round t and round t+1 respectively.

[0041] The present invention provides a federated learning system based on meta-learning and regularization, comprising:

[0042] Global model distribution module: The central server sends the global neural network parameters of the current iteration to the clients participating in the training. The global model parameters are generated by aggregating the update information uploaded by multiple clients in the previous iteration;

[0043] Global Model Local Adaptation Module: After receiving the global model parameters, the client performs two-stage training on the local data based on the meta-learning framework, including:

[0044] Inner loop update: divide the support set and query set from the local data, and use the support set to perform gradient update on the global model parameters to generate temporary parameters;

[0045] Outer loop optimization: Calculates the meta-gradients of temporary parameters based on the query set, adjusts global model parameters according to the preset learning rate, and generates client-specific update information;

[0046] Local model personalization module: Based on the global model parameters, the client optimizes the local model through regularization technology, including:

[0047] Calculate the L2 norm difference between the local model parameters and the global model parameters to generate the proximal term;

[0048] Combining the proximal term with the prediction loss of the local model to form an overall loss function;

[0049] Performing gradient updates on local model parameters based on the total loss function to generate a personalized local model;

[0050] Client model aggregation module: The central server receives the update information uploaded by each client, performs weighted mean aggregation on the update information according to a preset weight calculation formula, and generates the global model parameters for the next round of iteration.

[0051] In the above system, the two-stage meta-learning technology includes:

[0052] Support set partitioning: Client k samples tasks T from the local training set i , divide the support set based on task T and queryset

[0053] Inner loop parameter update: by formula:

[0054]

[0055] The learning rate η on the support set is based on the client's requirements g With the meta-training step size k inner Run gradient descent to generate temporary global model parameters;

[0056] where θ i+1 and θ i Respectively represent the temporary global model parameters corresponding to the local model when the step size is i and i+1 in the meta-training phase, Represents in task T i The global model θ on the support set i The training loss is represents θ i gradient;

[0057] Outer loop gradient calculation: by formula:

[0058]

[0059] Perform new global model update gradient calculation on the query set based on the temporary global model obtained in the meta-training phase;

[0060] where w t+1 and w t represent the global model before and after updating, β represents the learning rate in the meta-test phase, represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set, represents the loss function, Represents the global model parameter w t gradient.

[0061] In the above system, regularization technology:

[0062] The total loss function is:

[0063]

[0064] Where μ is the regularization strength, w t and Represent the local model and the global model respectively, Represents the actual loss value returned. The prediction error generated when the model makes prediction losses;

[0065] The gradient update formula is:

[0066]

[0067] where v k ′ and v k Represent the local personalized models before and after the update, Represents the gradient update information generated when the model predicts the loss, η l represents the local learning rate.

[0068] In the above system, the two-stage meta-learning technique includes: in weighted mean aggregation:

[0069] The weight calculation formula is:

[0070]

[0071] where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, D i represents the data volume of the i-th client, S t Represents the client number selected for the tth round of update;

[0072] The global model parameter update formula is:

[0073]

[0074] in, is the update amount of the global model by the i-th client in the t-th iteration, and They represent the global model parameters stored by the central server in round t and round t+1 respectively.

[0075] Because the present invention adopts the above technical means, it has the following beneficial effects:

[0076] 1. This invention addresses the technical issues of low global model convergence efficiency and weak generalization in traditional federated learning for non-independent and identically distributed (non-IID) data scenarios through the meta-learning two-stage optimization technique in step 2. By using a dual-gradient update mechanism for both the support set and the query set, the global model becomes more adaptable when acquiring local data features. This technique uses an inner loop to generate temporary parameters and an outer loop to perform meta-gradient calculations, ensuring that the global model update direction is more aligned with the local data distribution characteristics.

[0077] 2. The regularized proximal term constraint technology introduced in step 3 solves the problem of local optimization differences and personalized demand conflicts caused by the deviation of local model and global model parameters. The constraint mechanism enables the local model to maintain a controllable distance from the global knowledge system during the personalization process.

[0078] 3. The present invention combines the weighted mean aggregation mechanism in step 4 with the data volume weight calculation formula This technology solves the imbalance in model update weights caused by differences in client data volumes in traditional federated learning. By dynamically adjusting the aggregation coefficient, it ensures that the contribution value of clients with large amounts of data and clients with small amounts of data are fairly weighted.

[0079] 5. This invention solves the technical contradiction that a single global model is difficult to meet the personalized needs of clients through regularized gradient update technology. The constraint update mechanism enables the local model to follow the global knowledge guidance while maintaining personalized features. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a simplified diagram of the single-round distributed training process of the present invention;

[0081] Figure 2 It is the central server storage area and module architecture of the present invention;

[0082] Figure 3 The client server storage area and module architecture of the present invention. DETAILED DESCRIPTION

[0083] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with certain specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions of the present invention are intended to fall within the scope of the claims of the present invention.

[0084] In addition, in order to better illustrate the present invention, numerous specific details are given in the following detailed description. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0085] The present invention discloses a distributed neural network construction method that constructs a global neural network model based on a meta-learning framework and uses regularization technology to correct local modeling during the personalized adaptation stage of the local model. It mainly realizes a method for the neural network model to collaboratively construct a network model on clients with multiple data that are not interoperable, and mainly focuses on the fairness of the personalized utility of local clients and the discrimination level of local models. The construction method includes four steps: global model distribution, global model local adaptation, local model personalization, and client model aggregation. Global model distribution mainly provides global model parameter information to clients that need other model knowledge for local model tuning during the model construction process. The parameter information of the global model is aggregated through multiple client models, which can realize knowledge transfer between multiple clients with non-interoperable data. Global model local adaptation mainly focuses on how to make the global model adjust the global model parameters more efficiently, so as to realize the adaptation of the global model to local data, thereby obtaining the data characteristics of the local client. Local model personalization mainly solves the problem that a single global model cannot perfectly solve the decline in model accuracy when the client data has heterogeneous characteristics, while limiting the accuracy differences between local models as much as possible. Global model aggregation integrates the local data features captured by the global model, allowing the global model to integrate data from all clients. This construction method allows local models to be collaboratively constructed in the face of heterogeneous data, improving their accuracy while reducing the variance in accuracy between local models, even when data is not interoperable.

[0086] The present invention provides a distributed neural network construction method that builds a global neural network model based on a meta-learning framework and uses regularization technology to correct local modeling during the personalized adaptation stage of the local model. This method is characterized by dividing the model parameters generated by the local client into client-uploaded parameters for global model aggregation and client-local model parameters for local call. The client-uploaded parameters are globally constructed through meta-learning, and the client-local model parameters interact with the global model parameters distributed by the central server through regularization technology. The completion of the neural network model construction process requires the following parts and the implementation of the following functions:

[0087] Central Server:

[0088] Stores global model parameters aggregated from multiple clients during training;

[0089] Distribute parameters to clients that need global model parameters for training;

[0090] Aggregate global model update information from clients and form new global model parameters;

[0091] Client Server:

[0092] Stores client-collected data and local model parameters to protect data and model parameter privacy:

[0093] Perform meta-training of local data on the global model parameters issued, and upload update information containing client data information;

[0094] By using the distributed global model parameters, regularization techniques are used to personalize the local model.

[0095] The method of constructing a global neural network model based on a meta-learning framework and using regularization technology to correct the distributed neural network construction method of local modeling during the personalized adaptation stage of the local model is characterized in that meta-learning is used to accelerate the efficiency of extracting local client knowledge when constructing a global model, and the performance of the global model is improved by using the characteristics of two-stage derivation, thereby affecting the performance of the local model derived from the global model. When personalizing the local client, the proximal term is constructed through regularization technology to limit the gap between the local model and the global model to achieve interaction between the global model and the local model, thereby reducing the difference between the local models and meeting the fairness requirements of the client model.

[0096] The central server needs to have the following function: 1. The central server needs to have a secure storage medium that can cope with malicious attacks on the global model and a reliable storage mechanism to avoid leakage of the global model.

[0097] The central server needs to have function 2. The central server needs to build a complete communication mechanism to ensure the communication security between the central server and the client, avoid parameter leakage during transmission, and realize a safe and reliable data transmission mechanism.

[0098] The central server needs to have function 3, which is to aggregate the updated information generated by each round of client training, implement global model parameter updates, and complete the integration of client data information.

[0099] Function 1 that the client server needs to have is that the client server needs to have a secure large-capacity storage medium that can cope with malicious attacks against local personalized models and build a reliable storage mechanism that protects user data privacy and prevents local model information leakage.

[0100] Function 2 required by the client server is to complete the following steps when the local client updates the global model parameters:

[0101] S1: Divide the support set and query set required for meta-learning based on data size and user requirements;

[0102] S2: Implements the meta-training process in meta-learning. During the meta-training process, the global model parameters on the client are simulated and trained using the support set divided in S1. The learning rate and number of meta-training rounds in the meta-training process are determined according to requirements, and temporary global model parameter information is generated.

[0103] S3: Implements the meta-testing process in meta-learning. During the meta-testing process, the client uses the temporary global model parameter information generated in S2 to predict and discriminate the query set data divided in S1 to obtain the training loss on the query set, thereby obtaining the impact of the temporary update on the model prediction effect. Based on this impact, the original global model parameter information is adjusted to form the global model parameter update information;

[0104] S4: Upload the obtained global model parameter update information back to the central server, and request the server to aggregate the model update information.

[0105] Function 3 required of the client server is to complete the following steps when personalizing the local model on the local client:

[0106] S1: The client uses the local model to make predictions and judgments to obtain the training loss on the training dataset.

[0107] S2: The client's local model and the global model parameters distributed by the central server are calculated to obtain the proximal term;

[0108] S3: Through regularization technology, the sum of the proximal term and the training loss is used as the basis for updating the local model loss. The local model is gradient updated to obtain a new local personalized fairness model.

[0109] Example 1

[0110] Central Server

[0111] According to the attached Figure 2 As shown in the figure, the central server itself does not store any data, but only stores the global model generated during the training process. The generated global model can serve the neural network model construction itself, and also enable newly joined clients to fine-tune a high-quality local model based on the global model.

[0112] Transmission Module

[0113] The transmission module of the central server is mainly responsible for the distribution of global models and the collection of global model update information. For the sake of privacy protection, the model needs to be encrypted and decrypted before and after transmission to protect the privacy and security of the model.

[0114] The content that the transmission module needs to transmit is shown in Table 1:

[0115] Table 1 Transmission content of central server transmission module

[0116]

[0117] The global model parameters sent by the global model include the architecture of the neural network, the corresponding layers of each neural network, and the parameter data on the corresponding layers; the global model gradient in the global model update information refers to the difference between the new global model parameters generated during the local adaptation of the global model on the client and the original global model parameters, indicating the update direction and update step generated by the current global model on this client, and the client data volume represents how much data supports this update direction and update step, which is used to calculate the weight when aggregating update information.

[0118] Aggregation Module

[0119] The update method of the neural network needs to be data-driven. However, the central server itself does not generate and store data. The aggregation module implements a global model update method based on model parameters.

[0120] The aggregation module is based on weighted mean aggregation, and the weight calculation is as follows:

[0121]

[0122] where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, D i represents the data volume of the i-th client, S t Represents the client number selected for the tth round of update. The weight calculation formula ensures that:

[0123] The mean aggregation formula is as follows:

[0124]

[0125] where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, which is obtained by the weight calculation formula; and Represent the global model parameters stored in the central server in round t and round t+1 respectively, St Represents the client number selected for the tth round of update, Represents the adaptation of the local data of the i-th client in the t-th round to the global model The amount of change.

[0126] Client Server

[0127] According to the attached Figure 3 As shown in the figure, the client server stores local data and local models, and the neural network on the client is updated using the data-driven gradient descent method.

[0128] Transmission Module

[0129] The transmission module of the client server corresponds to the transmission module of the central server, and the transmission content is shown in Table 1. For privacy protection purposes, the client also needs to encrypt and decrypt the model before and after data transmission to protect the privacy of the model.

[0130] Global Adaptation Module

[0131] The local adaptation of the global model applies meta-learning techniques, and the corresponding steps are as follows:

[0132] Client k samples task T from the local training set i , divide the support set based on task T and queryset

[0133] The learning rate (η) on the support set is based on the client's requirements. g ) and meta-training step size (k inner ) Run the gradient descent method to generate a temporary global model. Its optimization formula corresponds to:

[0134]

[0135] where θ i+1 and θ i Respectively represent the temporary global model parameters corresponding to the local model when the step size is i and i+1 in the meta-training phase, η g represents the learning rate for temporary global model construction during the meta-training phase, Represents in task T i The global model θ on the support set i The training loss.

[0136] On the query set, a new global model update gradient calculation is performed based on the temporary global model obtained in the meta-training phase. The calculation formula is as follows:

[0137]

[0138] in and Represent the global model before and after the update, and β represents the learning rate in the meta-test phase. represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set.

[0139] where w t+1 and w t represent the global model before and after updating, β represents the learning rate in the meta-test phase, represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set.

[0140] After all tasks T are sampled in the local training set, update information is generated based on the final locally stored global model parameters and sent back to the central server.

[0141] Personalization module

[0142] Personalization adopts the method of proximal term update. The construction of proximal term adopts the L2 norm of the distance between the local model and the global model parameters. The proximal term formula is as follows:

[0143]

[0144] Where μ is the regularization strength, w t and Represent the local model and the global model respectively, Represents the actual loss value returned. Represents the prediction error generated when the model predicts the loss. The proximal term is derived in the gradient descent method, which changes the update formula in the gradient descent method to the following:

[0145]

[0146] where w t′ and w t Represent the local personalized model before and after the update, η l represents the learning rate during local personalization, Represents the gradient update information generated when the model predicts the loss, μ(v k -w t ) is the embodiment of the proximal term in the update formula, which linearly constrains the global model parameters and the local model parameter information, thereby limiting the gap between the two.

[0147] Algorithm Flow

[0148] The distributed neural network algorithm flow chart is as follows Figure 1 shown.

[0149] Central server process: After the central server starts, it first selects the client, then transmits the current global model parameters to the selected client, collects the global model update information of the selected server, performs mean aggregation on the global model to complete the update, and finally ends the process.

[0150] Client Process: The client begins by receiving the current global model parameters, generating global model updates using meta-learning, and then personalizing the local model using regularization. The process ends with the new local model. This entire process demonstrates the interaction and collaboration between the central server and the client on model updates.

Claims

1. A federated learning model optimization method based on meta-learning and regularization, characterized by: The following steps are involved: Step 1: Global model distribution: The central server sends the global neural network parameters of the current iteration to the clients participating in the training. The global model parameters are generated by aggregating the update information uploaded by multiple clients in the previous iteration; Step 2: Local adaptation of the global model: After receiving the global model parameters, the client performs two-stage training on the local data based on the meta-learning framework, including: Inner loop update: divide the support set and query set from the local data, and use the support set to perform gradient update on the global model parameters to generate temporary parameters; Outer loop optimization: Calculates the meta-gradients of temporary parameters based on the query set, adjusts global model parameters according to the preset learning rate, and generates client-specific update information; Step 3: Local model personalization: Based on the global model parameters, the client optimizes the local model using regularization techniques, including: Calculate the L2 norm difference between the local model parameters and the global model parameters to generate the proximal term; Combining the proximal term with the prediction loss of the local model to form an overall loss function; Performing gradient updates on local model parameters based on the total loss function to generate a personalized local model; Step 4: Client model aggregation: The central server receives the update information uploaded by each client, performs weighted mean aggregation on the update information according to a preset weight calculation formula, and generates the global model parameters for the next round of iteration.

2. The method according to claim 1, characterized in that The two-stage meta-learning technique includes: Support set partitioning: Client k samples tasks T from the local training set i , divide the support set based on task T and queryset Inner loop parameter update: by formula: The learning rate η on the support set is based on the client's requirements g With the meta-training step size k inner Run gradient descent to generate temporary global model parameters; where θ i+1 and θ i Respectively represent the temporary global model parameters corresponding to the local model when the step size is i and i+1 in the meta-training phase, Represents in task T i The global model θ on the support set i The training loss is represents θ i gradient; Outer loop gradient calculation: by formula: Perform new global model update gradient calculation on the query set based on the temporary global model obtained in the meta-training phase; in and represent the global model before and after updating, β represents the learning rate in the meta-test phase, represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set, represents the loss function, Represents the global model parameters gradient.

3. The method according to claim 1, characterized in that In the regularization technique of step 3: The total loss function is: Where μ is the regularization strength, w t and Represent the local model and the global model respectively, Represents the actual loss value returned. The prediction error generated when representing the model's prediction loss: The gradient update formula is: where v k ′ and v k Represent the local personalized models before and after the update, Represents the gradient update information generated when the model predicts the loss, η l represents the local learning rate.

4. The method according to claim 1, wherein The two-stage meta-learning technique includes: in the weighted mean aggregation of step 4: The weight calculation formula is: where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, D i represents the data volume of the i-th client, S t Represents the client number selected for the tth round of update; The global model parameter update formula is: in, is the update amount of the global model by the i-th client in the t-th iteration, and They represent the global model parameters stored by the central server in round t and round t+1 respectively.

5. A federated learning system based on meta-learning and regularization, characterized by: include: Global model distribution module: The central server sends the global neural network parameters of the current iteration to the clients participating in the training. The global model parameters are generated by aggregating the update information uploaded by multiple clients in the previous iteration; Global Model Local Adaptation Module: After receiving the global model parameters, the client performs two-stage training on the local data based on the meta-learning framework, including: Inner loop update: divide the support set and query set from the local data, and use the support set to perform gradient update on the global model parameters to generate temporary parameters; Outer loop optimization: Calculates the meta-gradients of temporary parameters based on the query set, adjusts global model parameters according to the preset learning rate, and generates client-specific update information; Local model personalization module: Based on the global model parameters, the client optimizes the local model through regularization technology, including: Calculate the L2 norm difference between the local model parameters and the global model parameters to generate the proximal term; Combining the proximal term with the prediction loss of the local model to form an overall loss function; Performing gradient updates on local model parameters based on the total loss function to generate a personalized local model; Client model aggregation module: The central server receives the update information uploaded by each client, performs weighted mean aggregation on the update information according to a preset weight calculation formula, and generates the global model parameters for the next round of iteration.

6. The system according to claim 5, characterized in that: The two-stage meta-learning technique includes: Support set partitioning: Client k samples tasks T from the local training set i , divide the support set based on task T and queryset Inner loop parameter update: by formula: The learning rate η on the support set is based on the client's requirements g With the meta-training step size k inner Run gradient descent to generate temporary global model parameters; where θ i+1 and θ i Respectively represent the temporary global model parameters corresponding to the local model when the step size is i and i+1 in the meta-training phase, Represents in task T i The global model θ on the support set i The training loss is represents θ i gradient; Outer loop gradient calculation: by formula: Perform new global model update gradient calculation on the query set based on the temporary global model obtained in the meta-training phase; where w t+1 and w t represent the global model before and after updating, β represents the learning rate in the meta-test phase, represents the training gradient generated by the temporary global model θ′ generated during the meta-training phase on the query set, represents the loss function, Represents the global model parameter w t gradient.

7. The system according to claim 5, characterized in that: Regularization techniques: The total loss function is: Where μ is the regularization strength, w t and Represent the local model and the global model respectively, Represents the actual loss value returned. The prediction error generated when representing the model's prediction loss: The gradient update formula is: where v k ′ and v k Represent the local personalized models before and after the update, Represents the gradient update information generated when the model predicts the loss, η l represents the local learning rate.

8. The method according to claim 5, characterized in that The two-stage meta-learning technique includes: in weighted mean aggregation: The weight calculation formula is: where λ i Represents the reliability of the update direction and update step represented by the current update information in the update information of the client participating in the tth round, D i represents the data volume of the i-th client, S t Represents the client number selected for the tth round of update; The global model parameter update formula is: in, is the update amount of the global model by the i-th client in the t-th iteration, and They represent the global model parameters stored by the central server in round t and round t+1 respectively.

Images