A personalized federated model training method based on meta-learning and multi-level segmentation

Abstract

The present invention relates to the field of artificial intelligence technology, and specifically discloses a personalized federated model training method based on meta-learning and multi-level segmentation. The present invention has a multi-level model segmentation architecture, which further subdivides the model into an input layer, an intermediate layer, and an output layer, thereby achieving a more refined distribution of computing load. Compared with the existing Split Learning method of dividing the model into two parts, the client and the server, the present invention localizes the input layer and the output layer, and retains the intermediate layer on the server side, which not only reduces the transmission of sensitive data but also balances computing efficiency and privacy protection. At the same time, the present invention has the ability to quickly fine-tune, so that the input layer and the output layer can dynamically adapt to the local data distribution of the client. The present invention achieves rapid adaptation of the local model through a meta-learning strategy, solves the problem of data heterogeneity, and significantly improves the personalized performance of the model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically discloses a personalized federated model training method based on meta-learning and multi-level segmentation. Background Art

[0002] Federated learning, as a distributed machine learning technology, aims to achieve joint modeling and improve the effectiveness of AI models while protecting data privacy and security and ensuring legal compliance.

[0003] In existing federated model training tasks, the client needs to send its local model to the server, which then aggregates the models received from all clients and sends the aggregated model to the client. This process is repeated until the model converges or the maximum number of iterations is reached.

[0004] However, when applied to data-sensitive and highly differentiated fields like healthcare and finance, federated learning's global aggregation mechanism struggles to cope with the varying distribution of client data. Furthermore, when balancing privacy protection with model performance, it often overlooks the need to dynamically adapt to local data characteristics.

[0005] The present invention provides a personalized federated model training method based on meta-learning and multi-level segmentation to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a personalized federated model training method that combines meta-learning with multi-level model segmentation. It solves the performance bottleneck problem caused by data heterogeneity through a fine segmentation architecture, a meta-learning-driven local fast fine-tuning module, and a hierarchical gradient refinement mechanism.

[0007] To achieve the above objectives, the basic solution of the present invention provides a personalized federated model training method based on meta-learning and multi-level segmentation, including the following steps:

[0008] Step S1: The server obtains the target model and divides it into an input layer, an intermediate layer, and an output layer to obtain a multi-level metamodel. The server initializes the parameters of the multi-level metamodel and distributes copies of the input layer and the output layer to each client, keeping the intermediate layer on the server.

[0009] Step S2: Each client constructs a local dataset with actual category labels, downloads input layer parameters and output layer parameters, uses local data in the local dataset as input samples for meta-learning fine-tuning, extracts preliminary features, and transmits them to the server to obtain intermediate layer activation values. The local data includes medical imaging data and corresponding category labels.

[0010] Step S3: The server calculates the global features based on the intermediate layer activation values ​​and normalizes them, updates the intermediate layer activation values ​​based on the normalized global features, and distributes them to each client;

[0011] Step S4: Each client generates a predicted classification result of the local data based on the updated intermediate layer activation value, and establishes an intermediate layer loss function based on the predicted classification result and the actual category label;

[0012] Step S5: Based on the intermediate layer loss function, each client calculates the gradients of the input layer parameters, intermediate layer parameters, and output layer parameters through back propagation to update the input layer parameters and output layer parameters;

[0013] Step S6: Each client uploads the gradient of the intermediate layer parameters to the server, which performs weighted average aggregation to update the global intermediate layer parameters and broadcasts them to all clients;

[0014] Step S7: Repeat steps S2 to S6 for iterative training until the model converges, and perform the target task through the converged model. The target task includes recognizing medical imaging data through the converged model to output the recognition result.

[0015] Further, in step S2, the local data set Build as follows:

[0016] ;

[0017] Where, Indicates the i Client's n input samples, Indicates the i Client's n The category labels of the input samples, Indicates the i The number of input samples in the local dataset of each client.

[0018] Further, in step S2, the meta-learning fine-tuning includes using the local dataset Perform K-step gradient descent, where K is between 1 and 5, and the larger the amount of local data, the larger the K value.

[0019] Furthermore, in step S2, the single-step gradient descent update process is expressed as follows:

[0020] ;

[0021] ;

[0022] Where, For the clienti Fine-tuning learning rate, For the client i The input layer parameters, For the client i The output layer parameters, For the client i The updated input layer parameters, For the client i The updated output layer parameters, For the client i The training loss, The calculation process is as follows:

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] Where, Represents the client i Upper input layer parameters The input layer function, Represents the client i Upper output layer parameters The output layer function, Represents the parameters of the middle layer The middle layer function of

[0028] Where, 、 and Respectively represent the client i Previous n The input layer results and intermediate layer results corresponding to the input samples: the intermediate layer activation value and the output layer result.

[0029] Furthermore, in step S3, the global feature is expressed as:

[0030] ;

[0031] Where, is the number of clients, Represents the client i Previous n The intermediate layer results corresponding to the input samples: the intermediate layer activation value, For the client i The weight of Indicates the i The number of input samples in the client's local dataset;

[0032] ;

[0033] Normalized global features The expression is as follows:

[0034] ;

[0035] Updated intermediate layer activation values The expression is as follows:

[0036] ;

[0037] Where, Update weights globally.

[0038] Further, in step S4, the classification result is predicted The expression is as follows:

[0039] ;

[0040] Where, Represents the client i Upper output layer parameters The output layer function of

[0041] The intermediate layer loss function is calculated as follows:

[0042] ;

[0043] In the formula, the second term is the L2 regularization term, is the regularization hyperparameter, is the actual category label.

[0044] Further, in step S5, the input layer parameters , middle layer parameters and output layer parameters The calculation formulas of the gradients are as follows:

[0045] ;

[0046] ;

[0047] ;

[0048] Where, Represents the client i Previous n The input layer results corresponding to the input samples.

[0049] Further, in step S6, the server receives the intermediate layer gradients uploaded by all clients. , and aggregate by weighted average to obtain the global gradient: ;

[0050] The server uses the aggregated global gradient to update the global intermediate layer parameters :

[0051] ;

[0052] Where, is the learning rate of server i.

[0053] The principle and effect of this solution are:

[0054] 1. Compared to existing technologies, this invention features a multi-layer model segmentation architecture, further subdividing the model into input, intermediate, and output layers, achieving more refined computational load distribution. Compared to existing Split Learning approaches that split the model into client-server components, this invention localizes the input and output layers, retaining the intermediate layers on the server side. This reduces the transmission of sensitive data while balancing computational efficiency and privacy protection.

[0055] 2. Compared with existing technologies, this invention offers rapid fine-tuning capabilities, enabling the input and output layers to dynamically adapt to the client's local data distribution. This invention utilizes a meta-learning strategy to rapidly adapt the local model, addressing data heterogeneity and significantly improving the model's personalized performance.

[0056] 3. Compared to existing technologies, this invention also features a hierarchical gradient update and aggregation mechanism: the client is responsible for updating the input and output layers, while the server aggregates the intermediate layer gradients to update the intermediate layers. This process iterates until the final input, output, and intermediate layers are obtained, converging the model. The converged model is then used to execute the target task. This weighted gradient processing approach optimizes communication efficiency (reducing redundant data transmission) while balancing global sharing with local personalized needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 A flowchart of a personalized federated model training method based on meta-learning and multi-level segmentation proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0060] A personalized federated model training method based on meta-learning and multi-level segmentation, implementing e.g. Figure 1 As shown, the following steps are included:

[0061] Step S1: The server obtains the target model and splits it into an input layer, an intermediate layer, and an output layer to create a multi-layer metamodel. The server initializes the parameters of the multi-layer metamodel and distributes copies of the input and output layers to each client, retaining the intermediate layer on the server. In this example, the pneumonia diagnosis model is used as the target model.

[0062] Step S2: Each client builds a local dataset with actual category labels ,Each client downloads the input layer parameters and output layer parameters of the ,multi-level meta-model respectively, uses the local data in the local dataset as ,input samples to perform meta-learning fine-tuning on the input layer parameters and ,output layer parameters, extracts preliminary features and transmits them to the server, ,obtains the intermediate layer activation value.

[0063] The local dataset Build as follows:

[0064] ;

[0065] Where, Indicates the i Client's n input samples, Indicates the i Client's n The category labels of the input samples, Indicates the i The number of input samples in the local dataset of each client.

[0066] In this embodiment, the local dataset is a private dataset, referring to the non-public data stored locally by each client. This includes local data: medical imaging data and corresponding category labels. By establishing a local dataset, client privacy is protected. During federated model training, the client only uploads intermediate features (not raw data), avoiding direct sharing of raw data and reducing the risk of privacy leaks.

[0067] In this embodiment, each client specifically represents the client used by the medical institution participating in the model training. For a client used by a medical institution, the label distribution of the local data used by it can also be used to represent the proportion of medical imaging data containing cancer cell images in X-rays, and the proportion of medical imaging data not containing cancer cell images in X-rays.

[0068] Download the input layer parameters of the metamodel on each client separately and output layer parameters After that, remember to download it to the client i The input layer parameters and output layer parameters are and Meta-learning fine-tuning involves using a local dataset Perform K-step gradient descent, where K is between 1 and 5, and the larger the amount of local data, the larger the K value.

[0069] Specifically, the single-step gradient descent update process is expressed as follows:

[0070] ;

[0071] ;

[0072] Where, For the client i Fine-tuning learning rate, For the client i The updated input layer parameters, For the client i The updated output layer parameters, For the client i The training loss is The calculation process is as follows:

[0073] ;

[0074] ;

[0075] ;

[0076] ;

[0077] Where, Represents the client i Upper input layer parameters The input layer function, Represents the client i Upper output layer parameters The output layer function, Represents the parameters of the middle layer The middle layer function of

[0078] Where, 、 and Respectively represent the client i Previous n The input layer results and intermediate layer results corresponding to the input samples: the intermediate layer activation value and the output layer result.

[0079] Step S3: The server calculates global features based on the activation values ​​of the intermediate layer It is then normalized, and the intermediate layer activation values ​​are updated based on the normalized global features and distributed to each client.

[0080] Specific, global features The formula is as follows:

[0081] ;

[0082] Where, is the number of clients, For the client i The weight is set according to the amount of local data, and the expression is as follows:

[0083] ;

[0084] The server further analyzes the global features Perform normalization to obtain normalized global features :

[0085] ;

[0086] Next, we use the normalized global features Activation value of the middle layer Update and obtain the updated intermediate layer activation value :

[0087] ;

[0088] Where, Update weights globally.

[0089] Step S4: The client activates the intermediate layer based on the received updated value And according to the output layer parameters Generate predicted classification results , based on the predicted classification results Establish an intermediate layer loss function with the actual category label .

[0090] Among them, the predicted classification results The calculation is based on the updated intermediate layer activation value, and the formula is as follows:

[0091] ;

[0092] The intermediate layer loss function is calculated as follows:

[0093] ;

[0094] In the formula, the second term is the L2 regularization term, is a regularization hyperparameter used to prevent overfitting, is the actual category label.

[0095] Step S5: Based on the intermediate layer loss function , client i Calculate the input layer parameters through backpropagation , middle layer parameters and output layer parameters The client updates the local parameters based on the obtained gradient.

[0096] Specifically, the input layer parameters , middle layer parameters and output layer parameters The calculation formulas of the gradients are as follows:

[0097] ;

[0098] ;

[0099] ;

[0100] Where, Represents the input layer parameters The gradient, Represents the parameters of the middle layer The gradient, Represents the output layer parameters gradient.

[0101] The client updates the local parameters based on the obtained gradient as follows:

[0102] ;

[0103] ;

[0104] in, For the client i The training learning rate.

[0105] Step S6: Each client uploads the gradient of the intermediate layer parameters to the server, and the server performs weighted average aggregation to obtain the gradient of the aggregated intermediate layer parameters. The server updates the global intermediate layer parameters based on the gradient of the aggregated intermediate layer parameters. And broadcast to all clients.

[0106] The server receives all intermediate layer gradients uploaded by the client , and aggregate by weighted average to obtain the global gradient:

[0107] ;

[0108] The server uses the aggregated global gradient to update the global intermediate layer parameters :

[0109] ;

[0110] Where, is the learning rate of server i.

[0111] Step S7: Repeat steps S2 to S6 for iterative training until the model converges, and use the converged model to perform the target task. The target task includes: using the converged model to recognize medical imaging data and output recognition results.

[0112] This step ensures that the global model can learn shared features from the local data distribution of all clients while avoiding directly sharing the original data and protecting privacy.

[0113] This invention features a multi-layered model segmentation architecture, further subdividing the model into input, intermediate, and output layers, achieving more refined computational load distribution. Compared to existing Split Learning approaches that split the model into client-server components, this invention localizes the input and output layers, retaining the intermediate layers on the server side. This reduces the transmission of sensitive data while balancing computational efficiency and privacy protection.

[0114] This invention has the ability to quickly fine-tune, enabling the input and output layers to dynamically adapt to the client's local data distribution. This invention uses a meta-learning strategy to achieve rapid adaptation of the local model, addressing the problem of data heterogeneity and significantly improving the model's personalized performance.

[0115] This invention also features a hierarchical gradient update and aggregation mechanism: the client updates the input and output layers, while the server aggregates the intermediate layer gradients to update the intermediate layers. This process iterates until the final input, output, and intermediate layers are obtained, converging the model. The converged model then executes the target task. This weighted gradient processing optimizes communication efficiency (reducing redundant data transmission) while balancing global sharing with local customization.

[0116] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A personalized federated model training method based on meta-learning and multi-level segmentation, characterized by: The steps include: Step S1: The server obtains the target model and divides it into an input layer, an intermediate layer, and an output layer to obtain a multi-level metamodel. The server initializes the parameters of the multi-level metamodel and distributes copies of the input layer and the output layer to each client, keeping the intermediate layer on the server. Step S2: Each client constructs a local dataset with actual category labels, downloads input layer parameters and output layer parameters, uses local data in the local dataset as input samples for meta-learning fine-tuning, extracts preliminary features, and transmits them to the server to obtain intermediate layer activation values. The local data includes medical imaging data and corresponding category labels. Step S3: The server calculates the global features based on the intermediate layer activation values ​​and normalizes them, updates the intermediate layer activation values ​​based on the normalized global features, and distributes them to each client; Step S4: Each client generates a predicted classification result of the local data based on the updated intermediate layer activation value, and establishes an intermediate layer loss function based on the predicted classification result and the actual category label; Step S5: Based on the intermediate layer loss function, each client calculates the gradients of the input layer parameters, intermediate layer parameters, and output layer parameters through back propagation to update the input layer parameters and output layer parameters; Step S6: Each client uploads the gradient of the intermediate layer parameters to the server, which performs weighted average aggregation to update the global intermediate layer parameters and broadcasts them to all clients; Step S7: Repeat steps S2 to S6 for iterative training until the model converges, and perform the target task through the converged model. The target task includes recognizing medical imaging data through the converged model to output the recognition result.

2. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 1, characterized in that: In step S2, the local data set Build as follows: ； Where, Indicates the i Client's n input samples, Indicates the i Client's n The category labels of the input samples, Indicates the i The number of input samples in the local dataset of each client.

3. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 2, characterized in that: In step S2, the meta-learning fine-tuning includes using the local dataset Perform K-step gradient descent, where K is between 1 and 5, and the larger the amount of local data, the larger the K value.

4. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 3 is characterized in that: In step S2, the single-step gradient descent update process is expressed as follows: ； ； Where, For the client i Fine-tuning learning rate, For the client i The input layer parameters, For the client i The output layer parameters, For the client i The updated input layer parameters, For the client i The updated output layer parameters, For the client i The training loss is The calculation process is as follows: ； ； ； ； Where, Represents the client i Upper input layer parameters The input layer function, Represents the client i Upper output layer parameters The output layer function, Represents the parameters of the middle layer The middle layer function of Where, 、 and Respectively represent the client i Previous n The input layer results and intermediate layer results corresponding to the input samples: the intermediate layer activation value and the output layer result.

5. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 1, characterized in that: In step S3, the global feature is expressed as: ； Where, is the number of clients, Represents the client i Previous n The intermediate layer results corresponding to the input samples: the intermediate layer activation value, For the client i The weight of Indicates the i The number of input samples in the client's local dataset; Normalized global features The expression is as follows: ； Updated intermediate layer activation values The expression is as follows: ； Where, Update weights globally.

6. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 5, characterized in that: In step S4, the classification result is predicted The expression is as follows: ； Where, Represents the client i Upper output layer parameters The output layer function of The intermediate layer loss function is calculated as follows: ； In the formula, the second term is the L2 regularization term, is the regularization hyperparameter, is the actual category label.

7. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 6, characterized in that: In step S5, the input layer parameters , middle layer parameters and output layer parameters The calculation formulas of the gradients are as follows: ； ； ； Where, Represents the client i Previous n The input layer results corresponding to the input samples.

8. The personalized federated model training method based on meta-learning and multi-level segmentation according to claim 7, characterized in that: In step S6, the server receives all the intermediate layer gradients uploaded by the client. , and aggregate by weighted average to obtain the global gradient: ; The server uses the aggregated global gradient to update the global intermediate layer parameters : ； Where, is the learning rate of server i.

Images