Heterogeneous decentralized federated learning method and system based on double-layer optimization

Abstract

The application provides a heterogeneous decentralized federated learning method and system based on double-layer optimization, and relates to the technical field of federated learning. Before multiple clients communicate, historical training communication data is extracted; a comprehensive utility value is determined accordingly, the multiple clients are paired two by two according to the comprehensive utility value, and an adaptive communication network is constructed; the two paired clients are respectively denoted as a first client and a second client, and a proxy model of the first client learns a local model of the first client; parameters of the proxy model of the first client are sent to a proxy model of the second client; and the local model of the second client learns the updated proxy model of the second client. Without changing the individualized structure of the local model, the system considers the historical interaction effect and is compatible with the model structure difference, and the adaptability of the system in a dynamic network environment and the model interaction flexibility are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of federated learning, in particular to a heterogeneous decentralized federated learning method and system based on double-layer optimization. BACKGROUND

[0002] Federated learning (FL) realizes data "availability but invisibility" between institutions through distributed training. Traditional decentralized federated learning (DFL) removes the central server, and clients directly collaborate through peer-to-peer (P2P) communication. Existing DFL adopts random connection (such as Def-KT), pre-defined rules (such as the exponential graph of ProxyFL) or static connection based on data similarity (such as CWT), ignoring network dynamics and data heterogeneity, and cannot adapt to dynamic network environments. In addition, model aggregation is achieved through model parameter averaging (such as AvgPush), but it requires that the client model architecture be consistent to implement parameter averaging, which cannot be compatible with personalized models and conflicts with the demand for personalized models in actual scenarios. SUMMARY

[0003] The problem to be solved by the present application is that traditional decentralized federated learning cannot adapt to dynamic network environments and conflicts with the demand for personalized models in actual scenarios.

[0004] To solve the above problems, in a first aspect, the present application provides a heterogeneous decentralized federated learning method based on double-layer optimization, comprising:

[0005] Before the plurality of clients communicate, historical training communication data is extracted;

[0006] According to the historical training communication data, a comprehensive utility value corresponding to the pairing of the current client and the remaining clients is determined, and the plurality of clients are paired two by two according to the comprehensive utility value to build an adaptive communication network, wherein the comprehensive utility value is used to represent the value of the client communication;

[0007] The first client's proxy model learns the first client's local model to align the outputs of the proxy model and the local model, wherein the two paired clients are denoted as the first client and the second client;

[0008] The parameters of the first client's proxy model are sent to the second client's proxy model to update the parameters of the second client's proxy model, wherein the historical training communication data includes the parameters of the first client's proxy model sent;

[0009] The second client's local model learns the updated proxy model of the second client to align the outputs of the local model and the updated proxy model.

[0010] Optionally, the determining, according to historical training communication data, of a comprehensive utility value corresponding to pairing of the current client with the rest of the clients, and the pairing, according to the comprehensive utility value, of the plurality of clients two by two to construct an adaptive communication network comprises:

[0011] determining, according to two times of historical training communication data before the current time, a cross-entropy loss function value of the local model of the current client after two times of training to perform a classification task, wherein the classification task comprises a handwritten digital image classification task;

[0012] determining, according to the cross-entropy loss function value of the two times of training, a task utility value;

[0013] determining, according to an actual communication time and a maximum tolerable time between the current client and the other client paired in the two times of training, a communication efficiency penalty coefficient;

[0014] determining, according to the task utility value and the communication efficiency penalty coefficient, a comprehensive utility value.

[0015] Optionally, the determining, according to the task utility value and the communication efficiency penalty coefficient, of a comprehensive utility value further comprises:

[0016] dynamically adjusting the comprehensive utility value of the current client according to a historical pairing communication frequency between the current client and the rest of the clients to determine a comprehensive utility correction value corresponding to pairing of the current client with the rest of the clients;

[0017] taking the comprehensive utility correction value as an edge weight of a full connection graph formed by all the clients to solve a maximum weight matching problem to determine a matching relationship between the plurality of clients.

[0018] Optionally, the task utility value is

[0019]

[0020] wherein, represents a task utility value corresponding to training of the client j receiving training communication data from the client i, represents a cross-entropy loss function of the classification task; and respectively represent a local model trained in the client j after completion of the t-1th round of communication and the tth round of communication; represents input data of the local model, and the input data comprises a handwritten digital image; represents a real label; represents a parameter of the local model after completion of the t-1th round of communication; represents a parameter of the local model after completion of the tth round of communication; t represents a communication round;

[0021] the communication efficiency penalty coefficient is

[0022]

[0023] wherein, represents the corresponding communication efficiency penalty coefficient of client j receiving training communication data from client i for training, represents the maximum tolerated time; represents the actual communication time; is an indicator function, when the actual communication time exceeds the maximum tolerated time , the value of the indicator function is 1, otherwise 0;

[0024] The comprehensive utility value is

[0025]

[0026] wherein, represents the corresponding independent utility value of client j receiving training communication data from client i for training, represents the corresponding independent utility value of client i receiving training communication data from client j for training, represents the corresponding comprehensive utility value of client j receiving training communication data from client i for training, represents the corresponding comprehensive utility value of client i receiving training communication data from client j for training.

[0027] Optionally, the comprehensive utility correction value is

[0028]

[0029] wherein, represents the corresponding comprehensive utility value of client j receiving training communication data from client i for training, represents the number of historical pairing communications between client j and client i, is a hyperparameter for controlling the exploration intensity; t represents the communication round; represents assigning the value calculated on the right side of the arrow to the variable on the left side of the arrow.

[0030] Optionally, the process of learning between the local model and the agent model in the client includes:

[0031] inputting the same data into the two models, taking the output of the learned model as the true label and the output of the learning model as the predicted label; wherein, when the agent model learns the local model, the agent model is the learning model and the local model is the learned model; when the local model learns the agent model, the local model is the learning model and the agent model is the learned model;

[0032] Based on the true and predicted labels, the loss function value is determined, and backpropagation guides the learning model to optimize the parameters so that the predicted labels are aligned with the true labels.

[0033] Optionally, the loss function during the mutual learning process between the client's local model and the proxy model is:

[0034]

[0035] in, This represents the local model within client j. This represents the loss function of the local model. This represents the cross-entropy loss function of the local model. Indicates the weighting coefficient. Represents the output of the local model Output of the proxy model KL divergence between them This represents the proxy model within client j. This represents the loss function of the proxy model. This represents the cross-entropy loss function of the proxy model. The output of the proxy model Output of the local model The KL divergence between them.

[0036] Optionally, heterogeneous decentralized federated learning methods based on two-layer optimization also include:

[0037] During the initial phase of operation, configure global initial parameters and set up the proxy model within the client.

[0038] Optionally, heterogeneous decentralized federated learning methods based on two-layer optimization also include:

[0039] During the initial client matching, multiple clients are randomly paired in pairs to collect initial training communication data between the clients.

[0040] Secondly, the present invention also provides a heterogeneous decentralized federated learning system based on two-layer optimization, which implements the heterogeneous decentralized federated learning method based on two-layer optimization as described above. The system includes a coordinator and multiple clients. The coordinator includes a training record memory and a communication party selector.

[0041] The training record storage device stores the training communication data of each client;

[0042] The coordinator is used to extract historical training communication data before the plurality of clients communicate, and is also used to determine a comprehensive utility value corresponding to pairing of a current client with the rest of the clients according to the historical training communication data, and to pair the plurality of clients two by two according to the comprehensive utility value to build an adaptive communication network, wherein the comprehensive utility value is used to represent the value of the clients communicating;

[0043] The two paired clients are respectively denoted as a first client and a second client; a proxy model of the first client learns a local model of the first client to align outputs of the proxy model and the local model; parameters of the proxy model of the first client are sent to a proxy model of the second client to update parameters of the proxy model of the second client, wherein the historical training communication data comprises the sent parameters of the proxy model of the first client; a local model of the second client learns the updated proxy model of the second client to align outputs of the local model and the updated proxy model.

[0044] The application provides a heterogeneous decentralized federated learning method and system based on double-layer optimization.

[0045] The construction process of the communication network is driven by the historical training communication data, so that the pairing decision can reflect the actual interaction effect instead of static rules, thereby effectively adapting to the dynamic changes of the network environment. Compared with the pre-defined rules or the static method of random connection in the prior art, the dynamic pairing mechanism driven by the historical data enables the communication network to respond to the changes in the network environment and the data heterogeneity. The proxy model is introduced as an intermediary carrier into the interaction process between the clients, and knowledge transfer is achieved through the output alignment mechanism, thereby avoiding the dependence of direct parameter exchange on the consistency of the model architecture and supporting the interaction of heterogeneous personalized models. Without changing the personalized structure of the local model, the system considers both the historical interaction effect and the model structure difference in the process of client matching and model interaction, thereby solving the problems of insufficient adaptability of the communication network and model heterogeneity conflict in the decentralized federated learning, and significantly improving the adaptability of the system in the dynamic network environment and the flexibility of model interaction. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief descriptions will be given below to the drawings needed to be used in the embodiments or prior art descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 A structure schematic diagram of a heterogeneous decentralized federated learning system based on double-layer optimization provided by the embodiments of the present application;

[0048] Figure 2 A flowchart illustrating a heterogeneous decentralized federated learning method based on two-layer optimization provided in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of a two-layer optimized data transmission provided in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0052] like Figure 1 As shown in the embodiment of this application, a heterogeneous decentralized federated learning system based on two-layer optimization is provided, including a coordinator and multiple clients. The coordinator includes a training record memory and a communication party selector.

[0053] The training record storage device stores the training communication data of each client.

[0054] A coordinator is used to extract historical training communication data before multiple clients communicate; it is also used to determine the comprehensive utility value corresponding to the pairing of the current client with other clients based on the historical training communication data, and to pair multiple clients in pairs according to the comprehensive utility value to construct an adaptive communication network, wherein the comprehensive utility value is used to characterize the value of the client communicating.

[0055] The two paired clients are referred to as the first client and the second client, respectively. The proxy model of the first client learns the local model of the first client to align the output of the proxy model with that of the local model. The parameters of the proxy model of the first client are sent to the proxy model of the second client to update the parameters of the proxy model of the second client. The historical training communication data includes the parameters of the proxy model of the first client that were sent. The local model of the second client learns the updated proxy model of the second client to align the output of the local model with that of the updated proxy model.

[0056] Specifically, the coordinator includes a training record storage and a communication party selector for managing the interaction process between clients, recording the historical communication behavior between clients to support subsequent decision-making. In addition, the coordinator can be assumed by any client or set up separately. The coordinator extracts historical training communication data before multiple clients communicate, which involves reading historical interaction records from storage media, filtering data of the latest communication round based on timestamps, and extracting historical information of specific objects by client identifier classification to achieve efficient use of historical experience. The comprehensive utility value corresponding to the pairing of the current client and the remaining clients is determined according to the historical training communication data, which can be calculated based on the model performance improvement amplitude, communication resource consumption, or data distribution similarity, such as comparing the accuracy rate change of the local model on the validation set, measuring the actual communication bandwidth occupancy, or calculating the cosine similarity of client data features, which is used to comprehensively reflect the potential contribution of pairing to system performance. According to the comprehensive utility value, multiple clients are paired two by two to build an adaptive communication network, and the pairing process can use a greedy algorithm to preferentially select high-value pairing, a graph theory-based maximum flow algorithm to optimize global matching, or introduce a random disturbance mechanism to avoid local optimal solution, to achieve dynamic adjustment of the communication topology. The two clients paired are referred to as the first client and the second client, respectively, and the proxy model of the first client learns the local model of the first client to align the outputs, so that the proxy model output approximates the local model output, by minimizing the mean square error of the predicted output, optimizing the feature space mapping relationship, or adjusting the model parameter similarity. The parameters of the proxy model of the first client are sent to the second client to update its proxy model, to achieve efficient knowledge transfer. The local model of the second client learns the updated proxy model to align the outputs, generates soft labels based on the proxy model output for supervised training, uses intermediate features in the proxy model to guide local model optimization, or enhances model robustness through adversarial training, for the fusion of external knowledge and the preservation of local characteristics.

[0057] In one example, the system is deployed in a distributed network consisting of 4 clients, and the coordinator calculates the comprehensive utility value based on the training data recorded in the past three rounds of communication. For example, when client A is paired with client B, the proxy model (recurrent neural network) of client A aligns with the output of the local model (convolutional neural network) by minimizing the KL divergence between them; the aligned proxy model parameters are sent to client B, and the proxy model (recurrent neural network) of client B updates the parameters accordingly; the local model (long short-term memory network) of client B then optimizes the output of the proxy model through a supervised learning process to ensure that the local model output is consistent with the proxy model output.

[0058] In the embodiment, the construction process of the communication network is driven by historical training communication data, so that the pairing decision can reflect the actual interaction effect instead of static rules, thereby effectively adapting to the dynamic changes of the network environment. Compared with the pre-defined rules or random connection of the static method in the traditional technical solution, the dynamic pairing mechanism driven by historical data enables the communication network to respond to network environment changes and data heterogeneity. The agent model is introduced as an intermediary carrier in the interaction process between clients, and knowledge transfer is realized through the output alignment mechanism, which avoids the dependence of direct parameter exchange on the consistency of the model architecture and supports the interaction of heterogeneous personalized models. Without changing the personalized structure of the local model, the system considers both the historical interaction effect and the compatibility of the model structure in the process of client matching and model interaction, thereby solving the problems of insufficient adaptability of the communication network and model heterogeneity conflict in the decentralized federated learning, and significantly improving the adaptability of the system in the dynamic network environment and the flexibility of model interaction.

[0059] As shown in Figure 2 , the embodiment of the application provides a heterogeneous decentralized federated learning method based on double-layer optimization, which comprises the following steps:

[0060] S1: Extracting historical training communication data before a plurality of clients communicate.

[0061] S2: According to the historical training communication data, determining the corresponding comprehensive utility value when the current client is paired with the remaining clients, and pairing the plurality of clients two by two according to the comprehensive utility value to construct an adaptive communication network, wherein the comprehensive utility value is used to represent the value of the client communication.

[0062] S3: Letting the agent model of the first client learn the local model of the first client to align the outputs of the agent model and the local model, wherein the two paired clients are respectively denoted as the first client and the second client.

[0063] S4: Sending the parameters of the agent model of the first client to the agent model of the second client to update the parameters of the agent model of the second client, wherein the historical training communication data comprises the parameters of the agent model of the first client.

[0064] S5: Letting the local model of the second client learn the updated agent model of the second client to align the outputs of the local model and the updated agent model.

[0065] In the embodiment, the extraction of the historical training communication data enables the system to predict the current communication effect based on past interaction experience, avoiding the blindness of random or predefined connection; the determination of the comprehensive utility value quantifies the task performance and communication efficiency as a unified index, ensuring that the pairing decision takes into account both model performance improvement and actual communication constraints; the construction of the adaptive communication network enables the topology structure to dynamically evolve with changes in data distribution and communication conditions; the output alignment mechanism of the agent model and the local model enables different architecture models to transfer knowledge through feature alignment, avoiding the dependence of parameter averaging on architecture consistency; the transmission and update of the agent model parameters enable safe and efficient information exchange between heterogeneous clients, ultimately fusing the collaboration benefits while preserving personalized features. By combining the dynamic pairing mechanism driven by historical training communication data with the knowledge transfer method mediated by the agent model, the network dynamic change and data heterogeneity challenges are effectively addressed, while supporting the training of heterogeneous personalized models, achieving improved adaptability of the system in dynamic network environments, and significantly enhancing model interaction flexibility.

[0066] The following describes each step in detail.

[0067] S1: Before the plurality of clients communicate, historical training communication data is extracted.

[0068] Specifically, an initial stage is run, global initial parameters (such as maximum communication delay time, etc.) are configured, agent models within the clients are set, and a client registry is managed. In the initial stage after the entire system architecture is built, since the system has just been put into use, global parameters need to be configured, agent models need to be set, parameter inconsistency problems caused by chaotic start order are avoided, initial bias caused by data distribution differences between clients is alleviated, and heterogeneous clients can carry out subsequent collaboration on a common benchmark. After running for a few rounds, historical training communication data is saved, which can be used to build a communication network using historical training communication data.

[0069] S2: According to the historical training communication data, determine the corresponding comprehensive utility value when the current client is paired with the remaining clients, and pair the plurality of clients two by two according to the comprehensive utility value to construct an adaptive communication network, wherein the comprehensive utility value is used to represent the value of the client in communication.

[0070] Specifically, each client first initializes the local model and the agent model in the initial stage of system running. In the subsequent running process of the system, after determining the communication party, point-to-point knowledge sharing with the communication party is realized through the double knowledge transfer method, and the training record of the current communication round is fed back to the coordinator.

[0071] When the client matching is first performed (i.e., the first round of communication), a plurality of clients are randomly paired two by two to collect initial training communication data between the clients. Randomly pairing a plurality of clients two by two can be achieved by an unbiased connection strategy of a pseudo-random number generator, so that all clients participate in communication equally without prior information and quickly establish initial connection relationships.

[0072] S21: According to the cross-entropy loss function value of the local model of the current client performing a classification task after two times of training according to the two times of historical training communication data before the current moment, wherein the classification task can be a handwritten digital image classification task.

[0073] S22: Determine the task utility value according to the cross-entropy loss function value of the two times of training. The task utility value is used to measure the performance improvement of the model obtained by the communication between the two clients. This index directly reflects the influence of the knowledge transfer of the client i in the tth round on the model performance of the client j. The larger the value is, the higher the task utility is. The task utility value is

[0074]

[0075] Wherein, represents the task utility value corresponding to the training of the client j receiving the training communication data from the client i, represents the cross-entropy loss function of the classification task; and respectively represent the local model trained in the client j after the completion of the t-1th round and the tth round of communication; represents the input data of the local model, and the input data includes handwritten digital images; represents the real label; represents the parameters of the local model after the completion of the t-1th round of communication; represents the parameters of the local model after the completion of the tth round of communication; t represents the communication round.

[0076] S23: Determine the communication efficiency penalty coefficient according to the actual communication time and the maximum tolerance time between the current client and the paired client in the two training processes. When the actual communication time required for the transmission of information between the client i and the client j in the tth round exceeds the maximum tolerance time , a power-law function based on the communication time is calculated, and the communication efficiency penalty coefficient is

[0077]

[0078] Wherein, represents the communication efficiency penalty coefficient corresponding to the training of the client j receiving the training communication data from the client i, denotes the maximum tolerated time; denotes the actual communication time; is an indicator function, when the actual communication time exceeds the maximum tolerated time , the value of the indicator function is 1, otherwise 0.

[0079] S24: Determine the comprehensive utility value according to the task utility value and the communication efficiency penalty coefficient. Take the product of the task utility value and the communication efficiency penalty coefficient as the comprehensive utility, considering , the final utility index is defined as the average value of the utilities of both parties, and the comprehensive utility value is

[0080]

[0081] wherein, denotes the independent utility value corresponding to the training of the client j receiving the training communication data from the client i, denotes the independent utility value corresponding to the training of the client i receiving the training communication data from the client j, denotes the comprehensive utility value corresponding to the training of the client j receiving the training communication data from the client i, denotes the comprehensive utility value corresponding to the training of the client i receiving the training communication data from the client j.

[0082] By first extracting the closest two historical training communication data before the current time as the input source, the cross-entropy loss function value of the local model executing the classification task after two trainings of the current client is calculated based on the data, which eliminates transient noise interference by using continuous historical data; then the cross-entropy loss function value is converted into a task utility value, which reflects the effectiveness of knowledge transfer between clients by quantifying the loss reduction amplitude; at the same time, the actual communication time and the preset maximum tolerated time in the two training processes are monitored, and a communication efficiency penalty coefficient is generated to represent the stability of network conditions; finally, the task utility value and the communication efficiency penalty coefficient are dynamically coupled to form a comprehensive utility value as the core basis for pairing decision. Due to the synergistic calculation mechanism of the task utility value and the communication efficiency penalty coefficient, the comprehensive utility value can reflect the model optimization potential and communication reliability at the same time, so as to preferentially select the client pairing combination with significant task performance improvement and controllable communication delay when building an adaptive communication network.

[0083] S25: dynamically adjusting the comprehensive utility value of the current client according to the historical pairing communication times between the current client and the remaining clients, determining the corresponding comprehensive utility correction value when the current client is paired with the remaining clients. In all subsequent rounds (t>1), both the potential training benefits of the communication parties and the collected communication data should be fully utilized. Adding an exploration reward to the known comprehensive utility value ensures that clients that have not been paired recently have a higher chance of being paired. The comprehensive utility correction value is

[0084]

[0085] wherein, represents the comprehensive utility value corresponding to the training of client j receiving training communication data from client i, represents the historical pairing communication times between client j and client i, is a hyperparameter that controls the exploration intensity; t represents the communication round; represents assigning the value calculated on the right side of the arrow to the variable on the left side of the arrow.

[0086] S26: taking the comprehensive utility correction value as the edge weight of the fully connected graph formed by all clients, solving the maximum weight matching problem to determine the matching relationship between multiple clients. After obtaining the comprehensive utility correction value, the coordinator can take it as the edge weight of the fully connected graph formed by all clients, solve the maximum weight matching problem, and determine the matching relationship between the clients.

[0087] During the operation of the federated learning system, when the coordinator calculates the comprehensive utility value of client A and client B, the system queries the training record storage to obtain the historical pairing communication times between the two, which is multiple times, and performs decay processing on the comprehensive utility value according to the preset rules; at the same time, the historical pairing times between client A and client C are less, and the corresponding comprehensive utility value is improved; taking the corrected comprehensive utility correction value between all clients as the edge weight of the fully connected graph, using the Hungarian algorithm to solve the maximum weight matching problem, finally determining the matching relationship that client A is paired with client C, and client B is paired with another client, realizing the dynamic optimization of the communication network, effectively avoiding the problem of fixed pairing relationship, dynamically balancing the historical interaction experience and the demand for new pairing exploration, significantly improving the adaptability of the communication network to dynamic environmental changes, preventing the system from falling into a local optimal pairing combination, enabling the communication network construction to dynamically balance historical experience and new connection exploration, significantly improving the ability to adapt to changes in client state in a dynamic network environment, and enhancing the robustness of the overall system.

[0088] The historical pairing communication times are used to dynamically adjust the comprehensive utility value, so that the combination utility value of the client pairing with frequent historical interactions is reduced to reduce the repetition tendency, and the combination utility value of the client pairing with rare historical interactions is improved to promote new pairing exploration, thereby introducing an exploration mechanism while retaining effective historical experience; then, the adjusted comprehensive utility correction value is mapped to the edge weight of the full connection graph, so that the edge weight comprehensively reflects the multi-dimensional factors of task utility, communication efficiency and historical interaction, and provides an accurate quantitative basis for the matching decision; finally, by solving the maximum weight matching problem, the client matching relationship is globally optimized under the constraint condition of pairwise pairing, ensuring the connectivity of the communication network and the efficiency of resource allocation, and forming a self-adaptive communication network construction mechanism that dynamically adapts to environmental changes.

[0089] S3: Let the agent model of the first client learn the local model of the first client, so that the outputs of the agent model and the local model are aligned, wherein the two paired clients are respectively denoted as the first client and the second client.

[0090] S4: Send the parameters of the agent model of the first client to the agent model of the second client to update the parameters of the agent model of the second client, wherein the historical training communication data comprises the parameters of the agent model of the first client.

[0091] S5: Let the local model of the second client learn the updated agent model of the second client, so that the outputs of the local model and the updated agent model are aligned.

[0092] Specifically, the same structure agent model is set in all clients, and the same structure agent model is used as an intermediary for knowledge transfer. The local model and the agent model are aligned in output by mutual learning, and the output distributions of the two models are aligned by KL divergence, so that the agent model can imitate the local model as much as possible, and the effect of subsequent cross-client knowledge transfer is guaranteed. As shown in Figure 3 The communication network construction stage (the specific steps include S1 and S2 described above) is regarded as upper optimization, and the coordinator dynamically adjusts the communication network to maximize the overall total utility by the adaptive communication network construction method based on the utility feedback of knowledge transfer. The knowledge transfer stage in the client (the specific steps are S3 described above) and the knowledge transfer stage between clients (the specific steps include S4 and S5 described above) are regarded as lower optimization, and the client minimizes the training loss (including task loss and KL divergence loss) of the local model by double knowledge transfer. In Figure 3 denotes the parameters of the local model in the client j, denotes the parameters of the agent model in the client j, denotes the parameters of the agent model in the client i, denotes the parameters of the local model in the client i. ​

[0093] The process of learning between the local model in the client and the agent model includes:

[0094] The same data is input to the two models, the output of the learned model is taken as the real label, and the output of the learning model is taken as the predicted label; wherein, when the agent model learns the local model, the agent model is the learning model, and the local model is the learned model; when the local model learns the agent model, the local model is the learning model, and the agent model is the learned model.

[0095] According to the real label and the predicted label, the loss function value is determined, and the learning model is guided to optimize the parameters in the reverse direction to align the predicted label with the real label.

[0096] The loss function in the mutual learning process of the local model and the agent model of the client is:

[0097]

[0098] wherein, represents the local model in the client j, represents the loss function of the local model, represents the cross-entropy loss function of the local model, represents the weighted coefficient, represents the output of the local model and the output of the agent model KL divergence between, represents the agent model in the client j, represents the loss function of the agent model, represents the cross-entropy loss function of the agent model, represents the output of the agent model and the output of the local model KL divergence between.

[0099] The dynamic alignment of the model output is realized by inputting the same data sample to the local model and the proxy model, assigning the output of the learned model as the real label, taking the output of the learned model as the predicted label, and optimizing the parameters of the learned model according to the loss function value through back propagation. In the learning of the local model by the proxy model, the proxy model receives the output of the local model as the real label, and aligns the output of the proxy model with the local model by minimizing the loss function; in the learning of the proxy model by the local model, the local model receives the output of the updated proxy model as the real label, and aligns the output of the local model with the proxy model by parameter optimization. The process is alternately executed in two directions, forming a closed-loop bidirectional knowledge transfer mechanism, ensuring that knowledge transfer can be realized only by output alignment in the case of heterogeneous model architecture, effectively adapting to the demand of personalized models and improving the robustness of the system, and improving the convergence performance and model generalization ability of federated learning in a dynamic network.

[0100] Exemplarily, in the experiment process, the client handles the handwritten digital image classification task, adopts the MNIST handwritten digital recognition data set, assumes that the client has 10, each client has 1000 MNIST images, and the category proportion is different; the local model in each client adopts different architectures (such as CNN, ResNet), and the proxy model is uniformly a 3-layer CNN network. The effect of the method of the application is compared with that of FedAvg, Def-KT, ProxyFL and the like.

[0101] Implementation steps: (1) initialization, the developer defines the proxy model as a 3-layer CNN, sets that each client only communicates with one client in each round of communication, and the maximum tolerated communication time is 10 seconds. (2) communication network construction and knowledge transfer interaction, wherein the coordinator updates the comprehensive utility according to the task utility feedback of the client and performs client pairing, and the client executes the pairing instruction and the knowledge transfer algorithm.

[0102] Experimental results: the average accuracy of multiple experiments of the method is 92.82%, which is improved by 2.17% compared with ProxyFL (90.65%). The communication time of the method is 491.54s, which is reduced by 18.1% compared with Def-KT (600.45s).

[0103] In summary, compared with the prior art, the following beneficial effects are possessed:

[0104] 1. A double-layer optimization framework is constructed, the upper-layer communication network optimization (maximizing utility) guides the lower-layer knowledge sharing (minimizing loss), and a feedback closed loop is formed. The performance is improved, and in the image classification task or other classification tasks, the accuracy is significantly improved, the parameter transmission amount and the communication time of the federated training process are reduced, and other key indicators are improved compared with the existing method.

[0105] 2. A dual knowledge transfer mechanism: The client internally distills local model knowledge into a standardized proxy model through KL divergence, and the clients exchange proxy models to solve the problem of heterogeneous model architecture. This supports the collaboration of clients with completely different model architectures (such as CNN, GRU, ResNet, etc.), breaking through the architectural limitations of traditional federated learning methods. In scenarios where the client's data is not independent and identically distributed (Non-IID), it can still achieve good performance improvement compared to local training and conventional federated learning methods.

[0106] 3. An adaptive node selection strategy dynamically selects communication partners based on a comprehensive utility (task utility + communication efficiency), balancing node diversity and communication costs, thus enhancing robustness. In dynamic network environments (i.e., extreme cases where the client network is unstable), adaptive selection can match suitable communication partners, ensuring the stable operation of the federated system model training.

[0107] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process or method that includes said element.

[0108] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A heterogeneous decentralized federated learning method based on two-layer optimization, characterized in that, The method, applied to the task of classifying handwritten digit images, includes: Extract historical training communication data before communication between multiple clients; Based on historical training communication data, the comprehensive utility value corresponding to the current client being paired with other clients is determined. Based on the comprehensive utility value, multiple clients are paired in pairs to construct an adaptive communication network. The comprehensive utility value is used to characterize the value of the client communicating. The proxy model of the first client learns the local model of the first client so that the output of the proxy model is aligned with the output of the local model. The two paired clients are referred to as the first client and the second client, respectively. The parameters of the proxy model of the first client are sent to the proxy model of the second client to update the parameters of the proxy model of the second client, wherein the historical training communication data includes the parameters of the proxy model of the first client that were sent. The local model of the second client learns the updated proxy model of the second client so that the output of the local model is aligned with the output of the updated proxy model. The step of determining the comprehensive utility value corresponding to pairing the current client with other clients based on historical training communication data, and then pairing multiple clients in pairs according to the comprehensive utility value to construct an adaptive communication network includes: Based on the two historical training communication data prior to the current moment, determine the cross-entropy loss function value of the local model performing the classification task after the current client's two training sessions; The task utility value is determined based on the cross-entropy loss function values ​​from the two training sessions. The communication efficiency penalty coefficient is determined based on the actual communication time and maximum tolerance time between the current client and its paired counterpart during the two training processes. The overall utility value is determined based on the task utility value and the communication efficiency penalty coefficient.

2. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 1, characterized in that, After determining the comprehensive utility value based on the task utility value and the communication efficiency penalty coefficient, the method further includes: Based on the historical number of pairing communications between the current client and other clients, the overall utility value of the current client is dynamically adjusted to determine the corresponding overall utility correction value when the current client is paired with other clients. By using the comprehensive utility correction value as the edge weight of the fully connected graph composed of all clients, the maximum weight matching problem is solved to determine the matching relationship between multiple clients.

3. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 1, characterized in that, The task utility value is in, This indicates that client j receives training communication data from client i to train the corresponding task utility value. The cross-entropy loss function represents the classification task; and These represent the local models trained on client j after the (t-1)th and tth rounds of communication, respectively. This represents the input data for the local model, which includes images of handwritten digits. Indicates the true label; This represents the parameters of the local model after the (t-1)th round of communication; This represents the parameters of the local model after the t-th round of communication; t represents the communication round. The communication efficiency penalty coefficient is: in, This represents the communication efficiency penalty coefficient corresponding to client j receiving training communication data from client i for training. Indicates the maximum tolerance time; Indicates the actual communication time; It is an indicator function, indicating the actual communication time. Exceeding the maximum tolerance time The value of the indicator function is 1 if the time is right, and 0 otherwise. The overall utility value is in, This indicates that client j receives training communication data from client i and uses it to train the corresponding independent utility value. This represents the independent utility value corresponding to the training data received by client i from client j. This represents the comprehensive utility value corresponding to the training data received by client j from client i. This represents the comprehensive utility value corresponding to the training data received by client i from client j.

4. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 2, characterized in that, The comprehensive utility correction value is in, This represents the comprehensive utility value corresponding to the training data received by client j from client i. This represents the number of historical pairing communications between client j and client i. It is a hyperparameter that controls the intensity of exploration; t represents the number of communication rounds; This means assigning the calculated value to the variable on the left side of the arrow.

5. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 1, characterized in that, The learning process between the local model and the proxy model within the client includes: When the same data is input into two models, the output of the model being learned is used as the true label, and the output of the model being learned is used as the predicted label. Specifically, when the surrogate model learns from the local model, the surrogate model is the learning model and the local model is the model being learned; when the local model learns from the surrogate model, the local model is the learning model and the surrogate model is the model being learned. Based on the true and predicted labels, the loss function value is determined, and backpropagation guides the learning model to optimize the parameters so that the predicted labels are aligned with the true labels.

6. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 5, characterized in that, The loss function during the mutual learning process between the client's local model and the proxy model is: in, This represents the local model within client j. This represents the loss function of the local model. This represents the cross-entropy loss function of the local model. Indicates the weighting coefficient. Represents the output of the local model Output of the proxy model KL divergence between them This represents the proxy model within client j. This represents the loss function of the proxy model. This represents the cross-entropy loss function of the proxy model. The output of the proxy model Output of the local model The KL divergence between them.

7. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 1, characterized in that, Also includes: During the initial phase of operation, configure global initial parameters and set up the proxy model within the client.

8. The heterogeneous decentralized federated learning method based on bi-layer optimization as described in claim 1, characterized in that, Also includes: During the initial client matching, multiple clients are randomly paired in pairs to collect initial training communication data between the clients.

9. A heterogeneous decentralized federated learning system based on two-layer optimization, characterized in that, The system is applied to the task of classifying handwritten digit images and implements the heterogeneous decentralized federated learning method based on bilayer optimization as described in any one of claims 1-8. The system includes a coordinator and multiple clients, and the coordinator includes a training record memory and a communication party selector. The training record storage device stores the training communication data of each client; A coordinator is used to extract historical training communication data before multiple clients communicate. It is also used to determine the comprehensive utility value corresponding to the pairing of the current client with other clients based on historical training communication data, and to pair multiple clients in pairs according to the comprehensive utility value to construct an adaptive communication network, wherein the comprehensive utility value is used to characterize the value of the client communicating; The two paired clients are referred to as the first client and the second client, respectively. The proxy model of the first client learns the local model of the first client to align the output of the proxy model with that of the local model. The parameters of the proxy model of the first client are sent to the proxy model of the second client to update the parameters of the proxy model of the second client. The historical training communication data includes the parameters of the proxy model of the first client that were sent. The local model of the second client learns the updated proxy model of the second client to align the output of the local model with that of the updated proxy model.

Images