Balanced federal learning personalization and generalization method and system based on super network

Abstract

The invention relates to a supernetwork-based balanced federal learning personalization and generalization method and system, and the method comprises the steps: carrying out the flexible switching of a personalization stage and a generalization stage, generating a generalization model through the aggregation of embedded vectors, and improving the overall generalization performance of the model; meanwhile, a personalized model is generated by using a client embedded vector, and the expression of the model under specific data distribution is enhanced. In the reasoning stage, a Shannon entropy-based dynamic selection mechanism is introduced, so that the accuracy and reliability of model reasoning under different data distributions are ensured, and the user experience is improved. In order to solve the problem of cross-client data isomerism in an edge intelligent scene, a hierarchical federated learning framework for generating a personalized model and a generalization model based on a super network is provided, and the framework can effectively balance personalized requirements and generalization requirements of an intelligent terminal.

Description

Technical Field

[0001] The present invention relates to a hypernetwork-based balanced federated learning personalization and generalization method and system, belonging to the technical field of federated learning. Background Art

[0002] With the widespread popularity of mobile devices and IoT devices and the continuous advancement of artificial intelligence technology, edge intelligence, as a new technology concept, has gradually become an important part of the digital age. In the industrial Internet of Things, edge machine learning can achieve real-time data analysis and predictive maintenance of equipment, thereby improving production efficiency and reducing operating costs; in the field of smart healthcare, relying on edge intelligence can achieve real-time health monitoring, intelligent diagnosis and personalized medical services, significantly improving the efficiency and quality of medical services.

[0003] As a distributed computing paradigm, edge computing aims to fully utilize the computing, storage, and communication resources of edge devices and comprehensively improve data processing capabilities by sinking computing, storage, and network resources from the cloud to the edge of the network closer to the source of data generation. As an effective supplement to traditional centralized cloud computing, edge computing has multiple advantages such as reducing the core network traffic load, relieving the pressure on cloud servers, shortening operation response delays, and improving data privacy and security. However, edge computing still faces some challenges in practical applications. On the one hand, when edge servers collect raw data from smart terminals in a centralized manner, there is still a risk of user privacy leakage. On the other hand, the quality and quantity of data in a single edge network may not be sufficient to support complex artificial intelligence algorithms, thus affecting the quality and reliability of edge intelligence services.

[0004] Federated learning is introduced into edge computing scenarios as a distributed learning method. It allows multiple participants to collaboratively train a shared global model without sharing original data, thereby ensuring the protection of participants' privacy data. In edge computing scenarios, each smart terminal device can use its own data set as well as computing, storage, and communication capabilities to collaborate with edge servers and cloud servers to train machine learning models, thereby achieving secure and high-quality edge intelligence. However, differences in sensing capabilities, environmental conditions, and user usage habits among different smart terminal devices result in significant heterogeneity in the collected data. The problem of data heterogeneity makes it difficult for the federated global model to adapt to the personalized needs and data characteristics of each client, thereby reducing the user experience.

[0005] In order to solve the problem of data heterogeneity across clients, personalized federated learning methods allow each client to use a personalized model instead of a shared global model, so as to better adapt to the local data distribution. However, personalized models usually rely on local data samples of the client for training, pay more attention to the distribution of local data, and lack the learning of global features. When the personalized model encounters data samples that have never appeared locally during the reasoning process, its misjudgment rate may be high. Therefore, in the process of achieving efficient and stable edge intelligent federated learning, how to strike a balance between the personalization and generalization of the model has become a crucial challenge. Summary of the invention

[0006] In order to solve the problem of client data heterogeneity in the federated learning scenario for edge intelligence, this paper proposes a hierarchical federated learning framework based on a hypernetwork to generate personalized models and generalized models. The solution is suitable for federated learning scenarios in edge computing scenarios where multiple IoT devices and smart terminal devices are jointly trained, and can achieve an effective balance between the personalized and generalized needs of edge intelligence.

[0007] Terminology explanation:

[0008] 1. Hypernetwork: Hypernetwork is a generative model proposed by David Ha and others. Its main idea is to use a generative network model to dynamically generate the weights of another target network model. The input of the hypernetwork is a set of embedding vectors that describe all the weights of a given layer, and it outputs the weight parameters required by the target network. This architecture allows the weights of the target network to be dynamically adjusted in different situations, thereby improving the flexibility and adaptability of the model. By generating the weights of the target network through the hypernetwork, the parameter storage requirements are reduced; by dynamically generating weights, the target network can be adjusted according to the input data or task requirements, and multiple target networks can share the same hypernetwork to achieve parameter sharing, thereby enhancing the adaptability and generalization ability of the model.

[0009] 2. FedAvg: FedAvg is a federated learning algorithm proposed by McMahan et al. Federated learning is a distributed machine learning paradigm in which multiple participants (such as mobile terminals, edge nodes, or data centers) collaborate to train a shared global model without transmitting the original data to a central server. Each client participating in the training task uses local data to train a local model and uploads the local model parameters after training to the central server. The central server aggregates the received model parameters by weighted averaging to obtain a global model. After multiple iterations of training, a shared global model that is close to the effect of centralized machine learning can be obtained. The algorithm aggregates the global model using local data on all devices to train the global model, which improves the accuracy and generalization performance of the model while effectively reducing the privacy leakage risks brought about by the collection of original training data in traditional centralized machine learning.

[0010] The technical solution of the present invention is:

[0011] A hypernetwork-based method for balancing personalization and generalization in federated learning, including:

[0012] Training phase:

[0013] First, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server;

[0014] Next, each edge server initializes the edge super network model and edge embedded network model according to the received global super network model and global embedded network model, i.e., the global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning;

[0015] Subsequently, each client initializes a local embedded network model, i.e., the edge embedded network model, based on the received edge embedded network model, generates an embedding vector of a local data sample using the local embedded network model, and feeds the embedding vector back to the edge server;

[0016] Each edge server generates a corresponding client model using the edge hypernetwork model according to the embedding vector fed back by the client. The client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client.

[0017] Each client uses local data samples to train the received client model, and calculates the updated parameters of the client model and feeds them back to the edge server;

[0018] Finally, each edge server updates parameters according to the client model fed back by each client. First, the gradient values ​​of the edge super network model are calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated using the aggregated gradient values. Second, the gradient values ​​of the embedding vectors of each client are calculated in parallel, and the updated parameters of the embedding vectors are calculated based on these gradient values, and the updated parameters are returned to the corresponding client. The client uses the received embedding vector to update the parameters, updates the local embedded network model, and feeds back the updated local embedded network model to the edge server. The edge server aggregates the local embedded network models fed back by all clients to obtain the updated edge embedded network model, completing a round of edge federated learning.

[0019] When each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, it uploads the latest edge super network model and edge embedded network model to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning;

[0020] Reasoning phase:

[0021] First, the client requests the edge server to embed the network model, uses the received embedding network model to generate the embedding vector of the local data sample, and feeds the embedding vector back to the edge server;

[0022] Next, the edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time;

[0023] Finally, the client uses the personalized model to calculate the Shannon entropy based on the local data samples that need to be inferred. If the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

[0024] Preferably, the specific implementation steps of the training phase are as follows:

[0025] (1.1) At the beginning of each round of global federated learning, the cloud server sends the global super network model, the global embedded network model, the global federated learning generalization training threshold, and the threshold of the number of local updates of the edge server to the edge server;

[0026] (1.2) The edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model;

[0027] (1.3) The edge server randomly selects some clients from the online clients within its range and sends the edge embedded network model to the selected clients;

[0028] (1.4) The client initializes the local embedding network model according to the received edge embedding network model, generates the embedding vector of the local data sample using the local embedding network model, and feeds the embedding vector back to the edge server;

[0029] (1.5) The edge server determines whether the number of global federated learning rounds reaches the generalization training threshold;

[0030] (1.5.1) If the set global federated learning generalization training threshold is not reached, then:

[0031] The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates the edge generalization model using the edge hypernetwork model, and sends the edge generalization model to the client;

[0032] The client uses local data samples to train the edge generalization model, calculates the edge generalization model update parameters, and feeds them back to the edge server;

[0033] (1.5.2) If the set global federated learning generalization training threshold is reached, then:

[0034] The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates an edge generalization model using the edge hypernetwork model, and saves the edge generalization model in the edge server;

[0035] The edge server generates a personalized model for each client based on the client's embedding vector using the edge hypernetwork model and sends the personalized model to the client;

[0036] The client uses local data samples to train the personalized model, calculates the personalized model update parameters, and feeds them back to the edge server;

[0037] (1.6) The edge server updates the parameters according to each client model. First, the gradient value of the edge super network model is calculated in parallel, the gradient values ​​of all clients are aggregated, and the aggregated gradient value is used to update the edge super network model; second, the gradient value of the client embedding vector is calculated in parallel, and the gradient value is used to calculate the embedding vector update parameter of each client, and the embedding vector update parameter is returned to the corresponding client;

[0038] (1.7) The client uses the received embedding vector to update the parameters and updates the local embedding network model, and feeds back the updated local embedding network model to the edge server. The edge server aggregates the local embedding network models fed back by all clients, completing a round of edge federated learning.

[0039] (1.8) When the edge server completes the update of the edge super network model and the edge embedded network model for a number of times exceeding the threshold number of local updates of the edge server, the latest edge super network model and the edge embedded network model are uploaded to the cloud server. The cloud server aggregates the received latest edge super network model and edge embedded network model to obtain a new global super network model and a global embedded network model, thus completing a round of global federated learning.

[0040] Preferably, the specific implementation steps of the reasoning stage are as follows:

[0041] (2.1) The client requests the edge server to embed the network model;

[0042] (2.2) The client generates an embedding vector for the local data sample using the received embedding network model and feeds the embedding vector back to the edge server;

[0043] (2.3) The edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time;

[0044] (2.4) The client calculates the Shannon entropy using the received personalized model based on the local data sample that needs to be inferred;

[0045] (2.5) The client determines whether the Shannon entropy meets the expected threshold:

[0046] (2.5.1) If the expected threshold is met, then:

[0047] The client uses the personalized model to perform reasoning tasks;

[0048] (2.5.2) If the expected threshold is not met, then:

[0049] The client uses the generalized model for reasoning tasks.

[0050] According to the preferred embodiment of the present invention, the client participating in the federated learning is set to C, and the edge server is set to Set the cloud server to S cloud ; M is the number of edge servers;

[0051] In step (1.1), the cloud server S cloud To edge server Send global hypernetwork model parameters Global Embedding Network Model Parameters Global federated learning generalization training threshold And the threshold of the number of local updates of the edge server δj , j∈[1,M]; t global represents the current round number of global federated learning, S cloud It refers to the cloud server; and δ j By cloud server S cloud Dynamic adjustment;

[0052] In step (1.2), the edge server According to the received global hypernetwork model parameters Global Embedding Network Model Parameters Initialize edge hypernetwork model parameters and edge embedding network model parameters t edge Indicates the current round number of edge federated learning, refers to edge server j;

[0053] In step (1.3), the edge server Randomly select some clients from N online clients within its range and record them as And to the client Broadcast edge embedding network model parameters

[0054] In step (1.4), the client Based on the received edge embedding network model parameters Initialize local embedding model network parameters Based on local data samples Randomly select a batch of data Use the local embedding network model to get the local embedding vector and will Feedback to edge server

[0055] According to the preferred embodiment of the present invention,

[0056] In step (1.5), the edge server Determine whether the number of global federated learning rounds reaches the generalization training threshold:

[0057] In step (1.5.1), if the number of global federated learning rounds does not reach the set generalization training threshold, that is, but:

[0058] Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector Agg() refers to the aggregation function;

[0059] Generate edge generalization model parameters using edge hypernetwork model

[0060] And to the client Broadcast edge generalization model parameters

[0061] Client According to the received edge generalization model parameters Initialize local model parameters Use local data samples to train the local model and optimize the objective function So as to obtain the updated local model parameters

[0062] Calculate local model update parameters Will Send to edge server

[0063] In step (1.5.2), if the number of global federated learning rounds reaches the set generalization training threshold, that is, but:

[0064] Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector Generate edge generalization model parameters using edge hypernetwork model And save this edge generalization model in the edge server;

[0065] Edge Server Based on the local embedding vector fed back by each client Generate personalized model parameters using edge hypernetwork model And to the client Send personalized model parameters

[0066] Client Use local data samples to train personalized models and optimize the objective function So as to obtain the updated local model parameters Calculate local model update parameters Will Send to edge server

[0067] According to the present invention, preferably, in step (1.6),

[0068] First, the edge server Update parameters according to each client model Parallel calculation of edge hypernetwork model gradient values Aggregate the edge hypernetwork model gradient values ​​for each client Using g h Update the edge super network model to obtain η represents the learning rate of the edge super network model, at which point a round of edge federated learning is completed and the edge super network model parameter update is completed;

[0069] Second, the edge server Update parameters according to each client model Parallel calculation of client embedding vector gradient values Embed the vector gradient value Send to client

[0070] According to the preferred embodiment of the present invention, in step (1.7), the client According to the received embedding vector gradient value Update the local embedded network model parameters, η represents the learning rate of the local embedding network model, and the updated local embedding network model parameters are obtained feedback To the edge server; the edge server receives all the local embedded network model parameters To aggregate, Embedding network model parameters for the next iteration of edges;

[0071] In step (1.8), the edge server Completion exceeds the set threshold δ j After the edge is updated, the latest edge super network model parameters and edge embedding network model parameters Upload to cloud server S cloud , Cloud Server S cloud For received and Use the FedAvg algorithm for aggregation to generate new global hypernetwork model parameters and the new global embedding network model parameters

[0072] According to the preferred embodiment of the present invention,

[0073] In step (2.1), the client To edge server Request embedding network model

[0074] In step (2.2), the client Based on the data samples that need to be inferred locally Refers to the local data set of the client that needs to perform reasoning tasks; a batch of data is randomly selected Use the received embedding network model to get the local embedding vector and will Feedback to edge server

[0075] In step (2.3), the edge server Local embedding vector based on client feedback Generate personalized model parameters using edge hypernetwork model And to the client Send personalized model parameters and the generalized model parameters stored in the edge server

[0076] In step (2.4), the client Based on the data samples that need to be inferred locally Calculate Shannon entropy using the received personalized model Q is the inference data sample The number of all possible categories that may appear when Is to use personalized model to sample The softmax output of the qth category, p (q) It refers to the output value of the softmax function, which is a probability value greater than 0 and less than 1;

[0077] In step (2.5), the client Determining Shannon Entropy Whether the expected threshold is met:

[0078] In step (2.5.1), if the expected threshold is met, that is but:

[0079] Client Using a Personalized Model Perform reasoning tasks;

[0080] In step (2.5.2), if the expected threshold is not met, that is, but:

[0081] Client Using the generalized model Perform reasoning tasks.

[0082] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned hypernetwork-based balanced federated learning personalization and generalization method are implemented.

[0083] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned hypernetwork-based balanced federated learning personalization and generalization method.

[0084] A balanced federated learning personalization and generalization system based on a hypernetwork, including:

[0085] The training module is configured as follows: first, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server; next, each edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning; then, each client initializes the local embedded network model according to the received edge embedded network model, uses the local embedded network model to generate the embedding vector of the local data sample, and feeds the embedding vector back to the edge server; each edge server uses the edge super network model to generate the corresponding client model according to the embedding vector fed back by the client, and the client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client; each client uses the local data sample to train the received client model, and calculates the client model update parameters and feeds them back to the edge server; finally, each edge server The server updates the parameters according to the client model fed back by each client; first, the gradient value of the edge super network model is calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated with the aggregated gradient value; second, the gradient value of the embedding vector of each client is calculated in parallel, and the update parameters of the embedding vector are calculated based on these gradient values, and the updated parameters are returned to the corresponding client; the client uses the received embedding vector to update the parameters, updates the local embedded network model, and feeds back the updated local embedded network model to the edge server; the edge server aggregates the local embedded network models fed back by all clients to obtain the updated edge embedded network model, completing a round of edge federated learning; when each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, the latest edge super network model and edge embedded network model are uploaded to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning;

[0086] The reasoning module is configured as follows: first, the client requests an embedded network model from the edge server, uses the received embedded network model to generate an embedding vector of a local data sample, and feeds the embedding vector back to the edge server; next, the edge server generates a corresponding personalized model using the edge super network model according to the embedding vector fed back by the client; and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time; finally, the client calculates the Shannon entropy based on the local data sample that needs to be inferred using the personalized model, and if the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

[0087] The beneficial effects of the present invention are:

[0088] 1. To address the problem of data heterogeneity across clients in edge intelligence scenarios, a hierarchical federated learning framework based on a hypernetwork to generate personalized models and generalized models is proposed. This framework can effectively balance the personalized and generalized needs of smart terminals.

[0089] 2. During the training phase, flexibly switch between the personalization and generalization phases, generate a generalization model by aggregating embedding vectors, and improve the overall generalization performance of the model; at the same time, use the client embedding vectors to generate a personalized model to enhance the performance of the model under specific data distribution.

[0090] 3. In the reasoning stage, a dynamic selection mechanism based on Shannon entropy is introduced to ensure the accuracy and reliability of model reasoning under different data distributions, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 This is a schematic diagram of the evaluation results of the personalized performance experiment;

[0092] Figure 2 This is a schematic diagram of the generalization performance experimental evaluation results;

[0093] Figure 3 This is a schematic diagram of the hierarchical federated learning architecture;

[0094] Figure 4 A schematic diagram of the interaction protocol for generating a client model; DETAILED DESCRIPTION

[0095] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0096] Example 1

[0097] A hypernetwork-based method for balancing personalization and generalization in federated learning, including:

[0098] The present invention proposes a hierarchical federated learning framework based on a hypernetwork to generate a personalized model and a generalized model. Figure 3 As shown in the figure, the solution is suitable for federated learning scenarios in edge computing scenarios where multiple IoT devices and smart terminal devices are jointly trained, and can achieve an effective balance between the personalized and generalized needs of edge intelligence.

[0099] Training phase:

[0100] First, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server;

[0101] Next, each edge server initializes the edge super network model and edge embedded network model according to the received global super network model and global embedded network model, i.e., the global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning;

[0102] Subsequently, each client initializes a local embedded network model, i.e., the edge embedded network model, based on the received edge embedded network model, generates an embedding vector of a local data sample using the local embedded network model, and feeds the embedding vector back to the edge server;

[0103] like Figure 4 As shown, each edge server generates a corresponding client model using the edge super network model according to the embedding vector fed back by the client, the client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client;

[0104] Each client uses local data samples to train the received client model, and calculates the updated parameters of the client model and feeds them back to the edge server;

[0105] Finally, each edge server updates parameters according to the client model fed back by each client. First, the gradient values ​​of the edge super network model are calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated using the aggregated gradient values. Second, the gradient values ​​of the embedding vectors of each client are calculated in parallel, and the updated parameters of the embedding vectors are calculated based on these gradient values, and the updated parameters are returned to the corresponding client. The client uses the received embedding vector to update the parameters, updates the local embedded network model, and feeds back the updated local embedded network model to the edge server. The edge server aggregates the local embedded network models fed back by all clients to obtain the updated edge embedded network model, completing a round of edge federated learning.

[0106] When each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, it uploads the latest edge super network model and edge embedded network model to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning;

[0107] Reasoning stage:

[0108] First, the client requests the edge server to embed the network model, uses the received embedding network model to generate the embedding vector of the local data sample, and feeds the embedding vector back to the edge server;

[0109] Next, the edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time;

[0110] Finally, the client uses the personalized model to calculate the Shannon entropy based on the local data samples that need to be inferred. If the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

[0111] Example 2

[0112] The hypernetwork-based balanced federated learning personalization and generalization method described in Example 1 is different in that:

[0113] The specific implementation steps of the training phase are as follows:

[0114] (1.1) At the beginning of each round of global federated learning, the cloud server sends the global super network model, the global embedded network model, the global federated learning generalization training threshold, and the threshold of the number of local updates of the edge server to the edge server;

[0115] (1.2) The edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model;

[0116] (1.3) The edge server randomly selects some clients from the online clients within its range and sends the edge embedded network model to the selected clients;

[0117] (1.4) The client initializes the local embedding network model according to the received edge embedding network model, generates the embedding vector of the local data sample using the local embedding network model, and feeds the embedding vector back to the edge server;

[0118] (1.5) The edge server determines whether the number of global federated learning rounds reaches the generalization training threshold;

[0119] (1.5.1) If the set global federated learning generalization training threshold is not reached, then:

[0120] The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates the edge generalization model using the edge hypernetwork model, and sends the edge generalization model to the client;

[0121] The client uses local data samples to train the edge generalization model, calculates the edge generalization model update parameters, and feeds them back to the edge server;

[0122] (1.5.2) If the set global federated learning generalization training threshold is reached, then:

[0123] The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates an edge generalization model using the edge hypernetwork model, and saves the edge generalization model in the edge server;

[0124] The edge server generates a personalized model for each client based on the client's embedding vector using the edge hypernetwork model and sends the personalized model to the client;

[0125] The client uses local data samples to train the personalized model, calculates the personalized model update parameters, and feeds them back to the edge server;

[0126] (1.6) The edge server updates the parameters according to each client model. First, the gradient value of the edge super network model is calculated in parallel, the gradient values ​​of all clients are aggregated, and the aggregated gradient value is used to update the edge super network model; second, the gradient value of the client embedding vector is calculated in parallel, and the gradient value is used to calculate the embedding vector update parameter of each client, and the embedding vector update parameter is returned to the corresponding client;

[0127] (1.7) The client uses the received embedding vector to update the parameters and updates the local embedding network model, and feeds back the updated local embedding network model to the edge server. The edge server aggregates the local embedding network models fed back by all clients, completing a round of edge federated learning.

[0128] (1.8) When the edge server completes the update of the edge super network model and the edge embedded network model for a number of times exceeding the threshold number of local updates of the edge server, the latest edge super network model and the edge embedded network model are uploaded to the cloud server. The cloud server aggregates the received latest edge super network model and edge embedded network model to obtain a new global super network model and a global embedded network model, thus completing a round of global federated learning.

[0129] During the training process of the global federated learning model, steps (1.1)-(1.8) are iteratively executed; during the training process of the edge federated learning model, steps (1.3)-(1.7) are iteratively executed.

[0130] The specific implementation steps of the reasoning phase are as follows:

[0131] (2.1) The client requests the edge server to embed the network model;

[0132] (2.2) The client generates an embedding vector for the local data sample using the received embedding network model and feeds the embedding vector back to the edge server;

[0133] (2.3) The edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time;

[0134] (2.4) The client calculates the Shannon entropy using the received personalized model based on the local data sample that needs to be inferred;

[0135] (2.5) The client determines whether the Shannon entropy meets the expected threshold:

[0136] (2.5.1) If the expected threshold is met, then:

[0137] The client uses the personalized model to perform reasoning tasks;

[0138] (2.5.2) If the expected threshold is not met, then:

[0139] The client uses the generalized model for reasoning tasks.

[0140] Set the client participating in federated learning to C and the edge server to Set the cloud server to S cloud ; M is the number of edge servers;

[0141] In step (1.1), the cloud server S cloud To edge server Send global hypernetwork model parameters Global Embedding Network Model Parameters Global federated learning generalization training threshold And the threshold of the number of local updates of the edge server δ j , j∈[1,M]; t global represents the current round number of global federated learning, S cloud It refers to the cloud server; and δ j By cloud server Scloud Dynamic adjustments; for example: If it is set to 500, global federated learning generalization training will be performed after 500 rounds of global federated learning personalized training. j If it is set to 5, the edge server will provide the cloud server with the latest edge embedded network model and edge super network model after every 5 local updates.

[0142] The global hypernetwork model and the global embedding network model may be currently widely used models, such as simple CNN and DNN.

[0143] In step (1.2), the edge server According to the received global hypernetwork model parameters Global Embedding Network Model Parameters Initialize edge hypernetwork model parameters and edge embedding network model parameters tedge indicates the current round number of edge federated learning. refers to edge server j;

[0144] The edge super network model and edge embedded network model can be currently widely used models, such as simple CNN and DNN. In step (1.3), the edge server Randomly select some clients from N online clients within its range and record them as And to the client Broadcast edge embedding network model parameters

[0145] In step (1.4), the client Based on the received edge embedding network model parameters Initialize local embedding model network parameters Based on local data samples Randomly select a batch of data Use the local embedding network model to get the local embedding vector X is the sample data, and y is the label corresponding to the sample data x. Local embedding vector It uses the local embedded network model Take sample data b as input and get the output; Feedback to edge server

[0146] In step (1.5), the edge server Determine whether the number of global federated learning rounds reaches the generalization training threshold:

[0147] In step (1.5.1), if the number of global federated learning rounds does not reach the set generalization training threshold, that is, but:

[0148] Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector Agg() refers to an aggregation function; for example, average aggregation, weighted average aggregation. It aggregates the local embedding vectors of all clients to obtain a unique edge embedding vector.

[0149] Generate edge generalization model parameters using edge hypernetwork model Specifically, the edge hypernetwork model is used and the edge embedding vector is used as input to obtain the edge generalization model parameters.

[0150] And to the client Broadcast edge generalization model parameters

[0151] Client According to the received edge generalization model parameters Initialize local model parameters Use local data samples to train the local model and optimize the objective function So as to obtain the updated local model parameters The objective function is the loss function, and the purpose of optimization is to train to get the appropriate Make F() minimum;

[0152] Calculate local model update parameters Will Send to edge server

[0153] The local model here is used for machine learning tasks such as image classification tasks.

[0154] In step (1.5.2), if the number of global federated learning rounds reaches the set generalization training threshold, that is, but:

[0155] Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector Generate edge generalization model parameters using edge hypernetwork model Specifically, it means: using the edge hypernetwork model and the edge embedding vector as input, the edge generalization model parameters are obtained. And this edge generalization model is saved in the edge server;

[0156] Edge Server Based on the local embedding vector fed back by each client Generate personalized model parameters using edge hypernetwork model Specifically, it means: using the edge hypernetwork model and the local embedding vector as input, the personalized model parameters are obtained and sent to the client. Send personalized model parameters

[0157] Client Use local data samples to train personalized models and optimize the objective function So as to obtain the updated local model parameters The objective function is the loss function, and the purpose of optimization is to train to get the appropriate Minimize F(). Calculate the local model update parameters Will Send to edge server

[0158] In step (1.6), first, the edge server Update parameters according to each client model Parallel calculation of edge hypernetwork model gradient values Aggregate the edge hypernetwork model gradient values ​​for each client Using g h Update the edge super network model to obtain η represents the learning rate of the edge super network model, at which point a round of edge federated learning is completed and the edge super network model parameter update is completed;

[0159] Second, the edge server Update parameters according to each client model Parallel calculation of client embedding vector gradient values Embed the vector gradient value Send to client

[0160] In step (1.7), the client According to the received embedding vector gradient value Update the local embedded network model parameters, η represents the learning rate of the local embedding network model, and the updated local embedding network model parameters are obtained feedback To the edge server; the edge server receives all the local embedded network model parameters To aggregate, Embedding network model parameters for the next iteration of edges;

[0161] In step (1.8), the edge server Completion exceeds the set threshold δ j After the edge is updated, the latest edge super network model parameters and edge embedding network model parameters Upload to cloud server S cloud , Cloud Server S cloud For received and Use the FedAvg algorithm for aggregation to generate new global hypernetwork model parameters and the new global embedding network model parameters Specifically, it means: using the FedAvg algorithm, aggregating all edge super network models and edge embedded network models to obtain a unique global super network model and a unique global embedded network model.

[0162] Each client Need to communicate with edge server The parameter communication process of multiple rounds of iterations, the edge server Need to connect to the cloud server S cloud The parameter communication process is carried out in multiple rounds of iterations.

[0163] In step (2.1), the client To edge server Request embedding network model

[0164] In step (2.2), the client Based on the data samples that need to be inferred locally Refers to the local data set of the client that needs to perform reasoning tasks; for example: pictures to be classified and identified, medical records to be diagnosed, etc. Randomly select a batch of data Use the received embedding network model to get the local embedding vector and will Feedback to edge server

[0165] In step (2.3), the edge server Local embedding vector based on client feedback Generate personalized model parameters using edge hypernetwork model And to the client Send personalized model parameters and the generalized model parameters stored in the edge server

[0166] In step (2.4), the client Based on the data samples that need to be inferred locally Calculate Shannon entropy using the received personalized model Q is the inference data sample The number of all possible categories that may appear when Is to use personalized model to sample The softmax output of the qth category, p (q) It refers to the output value of the softmax function, which is a probability value greater than 0 and less than 1;

[0167] In step (2.5), the client Determining Shannon Entropy Whether the expected threshold is met:

[0168] In step (2.5.1), if the expected threshold is met, that is but:

[0169] Client Using a Personalized Model Perform reasoning tasks; specifically, the client uses a personalized model based on the data samples that need to be processed locally, such as images, medical records, etc., and uses the data as input to obtain the output given by the model. This output is the result of reasoning.

[0170] In step (2.5.2), if the expected threshold is not met, that is, but:

[0171] Client Using the generalized model Perform reasoning tasks. Specifically, the client uses a generalized model based on the data samples that need to be processed locally, such as images, medical records, etc., and takes the data as input to obtain the output given by the model. This output is the result of reasoning.

[0172] The solution of the present invention is applicable to edge intelligence scenarios, such as smart medical care, industrial Internet of Things and other fields for federated learning. The hypernetwork model and embedded network model are, for example, the currently widely used simple CNN, DNN, etc. The generated generalized model or personalized model is, for example, the currently widely used simple CNN, DNN, LSTM, Transformer, etc., which can be used for image classification tasks, text generation tasks, etc.

[0173] For the solution of the present invention, the experiment sets the hypernetwork model as a three-layer fully connected network model, and the embedded network model as a simple convolutional neural network model with the same architecture as the client model. In view of the data heterogeneity across clients, 2 categories of data samples are randomly assigned to 100 clients on the public dataset Cifar10.

[0174] The experimental evaluation results show that when the degree of non-independent and identically distributed data between the clients participating in federated learning is too high, the personalized performance of this method can achieve the same accuracy as the personalized federated learning method, such as Figure 1 As shown in Figure 2, the generalization performance of this method is much higher than that of the personalized federated learning method and the FedAvg method. Figure 2 shown.

[0175] Example 3

[0176] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the hypernetwork-based balanced federated learning personalization and generalization method described in Example 1 or 2 are implemented.

[0177] Example 4

[0178] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the hypernetwork-based balanced federated learning personalization and generalization method described in Example 1 or 2 are implemented.

[0179] Example 5

[0180] A balanced federated learning personalization and generalization system based on a hypernetwork, including:

[0181] The training module is configured as follows: first, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server; next, each edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning; then, each client initializes the local embedded network model according to the received edge embedded network model, uses the local embedded network model to generate the embedding vector of the local data sample, and feeds the embedding vector back to the edge server; each edge server uses the edge super network model to generate the corresponding client model according to the embedding vector fed back by the client, and the client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client; each client uses the local data sample to train the received client model, and calculates the client model update parameters and feeds them back to the edge server; finally, each edge server The server updates the parameters according to the client model fed back by each client; first, the gradient value of the edge super network model is calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated with the aggregated gradient value; second, the gradient value of the embedding vector of each client is calculated in parallel, and the update parameters of the embedding vector are calculated based on these gradient values, and the updated parameters are returned to the corresponding client; the client uses the received embedding vector to update the parameters, updates the local embedded network model, and feeds back the updated local embedded network model to the edge server; the edge server aggregates the local embedded network models fed back by all clients to obtain the updated edge embedded network model, completing a round of edge federated learning; when each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, the latest edge super network model and edge embedded network model are uploaded to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning;

[0182] The reasoning module is configured as follows: first, the client requests an embedded network model from the edge server, uses the received embedded network model to generate an embedding vector of a local data sample, and feeds the embedding vector back to the edge server; next, the edge server generates a corresponding personalized model using the edge super network model according to the embedding vector fed back by the client; and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time; finally, the client calculates the Shannon entropy based on the local data sample that needs to be inferred using the personalized model, and if the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

Claims

1. A balanced federated learning personalization and generalization method based on a hypernetwork, characterized by: include: Training phase: First, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server; Next, each edge server initializes the edge super network model and edge embedded network model according to the received global super network model and global embedded network model, i.e., the global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning; Subsequently, each client initializes a local embedded network model, i.e., the edge embedded network model, based on the received edge embedded network model, generates an embedding vector of a local data sample using the local embedded network model, and feeds the embedding vector back to the edge server; Each edge server generates a corresponding client model using the edge hypernetwork model according to the embedding vector fed back by the client. The client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client. Each client uses local data samples to train the received client model, and calculates the updated parameters of the client model and feeds them back to the edge server; Finally, each edge server updates the parameters according to the client model fed back by each client. First, the gradient values ​​of the edge super network model are calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated using the aggregated gradient values. Secondly, the gradient values ​​of the embedding vectors of each client are calculated in parallel, and the update parameters of the embedding vectors are calculated based on these gradient values, and the updated parameters are returned to the corresponding client; The client uses the received embedding vector to update the parameters, update the local embedding network model, and feed back the updated local embedding network model to the edge server; the edge server aggregates the local embedding network models fed back by all clients to obtain the updated edge embedding network model, completing a round of edge federated learning; When each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, it uploads the latest edge super network model and edge embedded network model to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning; Reasoning phase: First, the client requests the edge server to embed the network model, uses the received embedding network model to generate the embedding vector of the local data sample, and feeds the embedding vector back to the edge server; Next, the edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time; Finally, the client uses the personalized model to calculate the Shannon entropy based on the local data samples that need to be inferred. If the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

2. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: The specific implementation steps of the training phase are as follows: (1.1) At the beginning of each round of global federated learning, the cloud server sends the global super network model, the global embedded network model, the global federated learning generalization training threshold, and the threshold of the number of local updates of the edge server to the edge server; (1.2) The edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model; (1.3) The edge server randomly selects some clients from the online clients within its range and sends the edge embedded network model to the selected clients; (1.4) The client initializes the local embedding network model according to the received edge embedding network model, generates the embedding vector of the local data sample using the local embedding network model, and feeds the embedding vector back to the edge server; (1.5) The edge server determines whether the number of global federated learning rounds reaches the generalization training threshold; (1.5.1) If the set global federated learning generalization training threshold is not reached, then: The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates the edge generalization model using the edge hypernetwork model, and sends the edge generalization model to the client; The client uses local data samples to train the edge generalization model, calculates the edge generalization model update parameters, and feeds them back to the edge server; (1.5.2) If the set global federated learning generalization training threshold is reached, then: The edge server aggregates the embedding vectors of all clients to obtain the edge embedding vector; generates an edge generalization model using the edge hypernetwork model, and saves the edge generalization model in the edge server; The edge server generates a personalized model for each client based on the client's embedding vector using the edge hypernetwork model and sends the personalized model to the client; The client uses local data samples to train the personalized model, calculates the personalized model update parameters, and feeds them back to the edge server; (1.6) The edge server updates the parameters according to each client model. First, the gradient value of the edge super network model is calculated in parallel, the gradient values ​​of all clients are aggregated, and the aggregated gradient value is used to update the edge super network model; second, the gradient value of the client embedding vector is calculated in parallel, and the gradient value is used to calculate the embedding vector update parameter of each client, and the embedding vector update parameter is returned to the corresponding client; (1.7) The client uses the received embedding vector to update the parameters and updates the local embedding network model, and feeds back the updated local embedding network model to the edge server. The edge server aggregates the local embedding network models fed back by all clients, completing a round of edge federated learning. (1.8) When the edge server completes the update of the edge super network model and the edge embedded network model for a number of times exceeding the threshold number of local updates of the edge server, the latest edge super network model and the edge embedded network model are uploaded to the cloud server. The cloud server aggregates the received latest edge super network model and edge embedded network model to obtain a new global super network model and a global embedded network model, thus completing a round of global federated learning.

3. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: The specific implementation steps of the reasoning phase are as follows: (2.1) The client requests the edge server to embed the network model; (2.2) The client generates an embedding vector for the local data sample using the received embedding network model and feeds the embedding vector back to the edge server; (2.3) The edge server generates a corresponding personalized model using the edge hypernetwork model based on the embedding vector fed back by the client, and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time; (2.4) The client calculates the Shannon entropy using the received personalized model based on the local data sample that needs to be inferred; (2.5) The client determines whether the Shannon entropy meets the expected threshold: (2.5.1) If the expected threshold is met, then: The client uses the personalized model to perform reasoning tasks; (2.5.2) If the expected threshold is not met, then: The client uses the generalized model for reasoning tasks.

4. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: Set the client participating in federated learning to C and the edge server to j∈[1,M], set the cloud server to S cloud ; M is the number of edge servers; In step (1.1), the cloud server S cloud To edge server Send global hypernetwork model parameters Global Embedding Network Model Parameters Global federated learning generalization training threshold And the threshold δ of the number of local updates of the edge server j ,J∈[1,M]; t global represents the current round number of global federated learning, S cloud It refers to the cloud server; and δ j By cloud server S cloud Dynamic adjustment; In step (1.2), the edge server According to the received global hypernetwork model parameters Global Embedding Network Model Parameters Initialize edge hypernetwork model parameters and edge embedding network model parameters t edge Indicates the current round number of edge federated learning, refers to edge server j; In step (1.3), the edge server Randomly select some clients from N online clients within its range and record them as And to the client Broadcast edge embedding network model parameters In step (1.4), the client Based on the received edge embedding network model parameters Initialize local embedding model network parameters Based on local data samples Randomly select a batch of data Use the local embedding network model to get the local embedding vector and will Feedback to edge server 5. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: In step (1.5), the edge server Determine whether the number of global federated learning rounds reaches the generalization training threshold: In step (1.5.1), if the number of global federated learning rounds does not reach the set generalization training threshold, that is but: Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector i∈[1,K], Agg() refers to the aggregation function; Generate edge generalization model parameters using edge hypernetwork model And to the client Broadcast edge generalization model parameters Client According to the received edge generalization model parameters Initialize local model parameters Use local data samples to train the local model and optimize the objective function So as to obtain the updated local model parameters Calculate local model update parameters Will Send to edge server In step (1.5.2), if the number of global federated learning rounds reaches the set generalization training threshold, that is, but: Edge Server Aggregate all clients Feedback local embedding vector Get the edge embedding vector Generate edge generalization model parameters using edge hypernetwork model And save this edge generalization model in the edge server; Edge Server Based on the local embedding vector fed back by each client Generate personalized model parameters using edge hypernetwork model And to the client Send personalized model parameters Client Use local data samples to train personalized models and optimize the objective function So as to obtain the updated local model parameters Calculate local model update parameters Will Send to edge server 6. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: In step (1.6), first, the edge server Update parameters according to each client model Parallel calculation of edge hypernetwork model gradient values Aggregate the edge hypernetwork model gradient values ​​for each client Using g h Update the edge super network model to obtain η represents the learning rate of the edge super network model, at which point a round of edge federated learning is completed and the edge super network model parameter update is completed; Second, the edge server Update parameters according to each client model Parallel computation of client embedding vector gradients Embed the vector gradient value Send to client 7. The method for balanced federated learning personalization and generalization based on a hypernetwork according to claim 1, characterized in that: In step (1.7), the client According to the received embedding vector gradient value Update the local embedded network model parameters, η represents the learning rate of the local embedding network model, and the updated local embedding network model parameters are obtained feedback To edge servers; The edge server receives all the local embedded network model parameters To aggregate, Embedding network model parameters for the next iteration of edges; In step (1.8), the edge server Completion exceeds the set threshold δ j After the edge is updated, the latest edge super network model parameters and edge embedding network model parameters Upload to cloud server S cloud , Cloud Server S cloud For received and Use the FedAvg algorithm for aggregation to generate new global hypernetwork model parameters and the new global embedding network model parameters Further preferably, in step (2.1), the client To edge server Request embedding network model In step (2.2), the client Based on the data samples that need to be inferred locally Refers to the local data set of the client that needs to perform reasoning tasks; a batch of data is randomly selected Use the received embedding network model to get the local embedding vector and will Feedback to edge server In step (2.3), the edge server Local embedding vector based on client feedback Generate personalized model parameters using edge hypernetwork model And to the client Send personalized model parameters and the generalized model parameters stored in the edge server In step (2.4), the client Based on the data samples that need to be inferred locally Calculate Shannon entropy using the received personalized model Q is the inference data sample The number of all possible categories that may appear when Is to use personalized model to sample The softmax output of the qth category, p (a) It refers to the output value of the softmax function, which is a probability value greater than 0 and less than 1; In step (2.5), the client Determining Shannon Entropy Whether the expected threshold is met: In step (2.5.1), if the expected threshold is met, that is but: Client Using a Personalized Model Perform reasoning tasks; In step (2.5.2), if the expected threshold is not met, that is, but: Client Using the generalized model Perform reasoning tasks.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the hypernetwork-based balanced federated learning personalization and generalization method described in any one of claims 1-7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hypernetwork-based balanced federated learning personalization and generalization method described in any one of claims 1 to 7 are implemented.

10. A balanced federated learning personalization and generalization system based on a hypernetwork, characterized by: include: The training module is configured as follows: first, at the beginning of each round of global federated learning, the cloud server sends the global super network model and the global embedded network model to each edge server; next, each edge server initializes the edge super network model and the edge embedded network model according to the received global super network model and the global embedded network model, and sends the edge embedded network model to the clients within the range of each edge server, and each edge network starts edge federated learning; Subsequently, each client initializes a local embedded network model according to the received edge embedded network model, uses the local embedded network model to generate an embedding vector of a local data sample, and feeds the embedding vector back to the edge server; each edge server uses the edge super network model to generate a corresponding client model according to the embedding vector fed back by the client, and the client model includes a personalized model or a generalized model, and sends the generated client model to the corresponding client; each client trains the received client model using the local data sample, and calculates the client model update parameters and feeds them back to the edge server; finally, each edge server updates the parameters according to the client model fed back by each client; first, the gradient values ​​of the edge super network model are calculated in parallel, the gradient values ​​of all clients are aggregated, and the edge super network model is updated using the aggregated gradient values; Secondly, the gradient values ​​of the embedding vectors of each client are calculated in parallel, and the update parameters of the embedding vectors are calculated based on these gradient values, and the updated parameters are returned to the corresponding client; The client uses the received embedding vector to update the parameters, updates the local embedded network model, and feeds back the updated local embedded network model to the edge server; the edge server aggregates the local embedded network models fed back by all clients to obtain the updated edge embedded network model, completing a round of edge federated learning; when each edge server completes the update of the edge super network model and edge embedded network model for the set threshold number of times, it uploads the latest edge super network model and edge embedded network model to the cloud server; the cloud server aggregates the received edge super network model and edge embedded network model to generate a new global super network model and global embedded network model, completing a round of global federated learning; The reasoning module is configured as follows: first, the client requests an embedded network model from the edge server, uses the received embedded network model to generate an embedding vector of a local data sample, and feeds the embedding vector back to the edge server; next, the edge server generates a corresponding personalized model using the edge super network model according to the embedding vector fed back by the client; and sends the generated personalized model and the generalized model stored in the edge server to the client at the same time; finally, the client calculates the Shannon entropy based on the local data sample that needs to be inferred using the personalized model, and if the Shannon entropy meets the expected threshold, the personalized model is used for the reasoning task; if the Shannon entropy does not meet the expected threshold, the generalized model is used for the reasoning task.

Images