Decentralized federated learning method and system based on structure and data perception, and storage medium

Abstract

The invention discloses a decentralized federated learning method and system based on structure and data perception, and a storage medium. The method comprises the following steps: initializing local model parameters by each node; in each round of iteration, each node collects model parameters from a neighbor node set and performs aggregation according to a Gossip matrix to obtain an intermediate model; constructing a decentralized learning-oriented contrast learning loss function; calculating a final local training loss function according to the degree centrality of the node; according to a local training loss function, carrying out one-time or multi-time gradient updating on the intermediate model obtained by aggregation on a local data set; and updating local model parameters according to the calculated gradient. According to the method, the problem of double influences of the existing decentralized federated learning network structure and data characteristic coupling on the model performance is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of information security, and particularly relates to a decentralized federated learning method and system based on structure and data perception and a storage medium. BACKGROUND

[0002] Under the background of increasingly strict data privacy protection regulations, federated learning (FL) as a new type of distributed learning paradigm has gradually become a key method for building edge intelligent systems. The core idea is that each participating node independently trains a model locally and periodically shares model parameters to collaboratively build a global model without centralizing raw data, thereby realizing intelligent modeling across devices while ensuring data security. Traditional federated learning architecture relies on a centralized parameter server model, that is, a central server collects local model parameters uploaded by each node and aggregates them. However, this architecture has obvious limitations: on the one hand, the failure or communication congestion of the central node can lead to a decline in the overall performance of the system; on the other hand, the central node may become a risk point for privacy leakage or security attacks in actual deployment. Therefore, this centralized structure is difficult to adapt to large-scale, dynamically changing, or security-sensitive network environments.

[0003] To solve the above problems, decentralized federated learning (DFL) has emerged in recent years. This method establishes an adjacent communication structure between multiple peer nodes to realize a centerless coordinated training process, avoiding dependence on a central server and being more suitable for dynamic and distributed application scenarios such as self-organizing networks and vehicle edge systems. However, in actual systems, good communication quality does not necessarily mean high data quality. Some nodes may have strong network connectivity and frequent participation, but their local data may have high noise or large distribution bias, which can mislead the global model. On the other hand, nodes with weak connectivity may have good data quality, but they cannot fully participate in training due to limited communication opportunities, leading to model bias. Therefore, existing methods have not effectively solved the problem of the dual influence of network structure and data characteristics on model performance.

[0004] To solve the problem of the dual influence of network structure and data characteristics on model performance, a decentralized federated learning method and system based on structure and data perception and a storage medium are proposed. SUMMARY

[0005] The present application proposes a decentralized federated learning method and system based on structure and data perception and a storage medium to at least solve the problem of the dual influence of network structure and data characteristics on model performance in existing decentralized federated learning.

[0006] According to one embodiment of the present application, a structure and data perception-based decentralized federated learning method is provided, comprising the steps of:

[0007] each node initializes local model parameters;

[0008] In each iteration, each node collects model parameters from its neighbor node set and aggregates them according to the Gossip matrix to obtain an intermediate model;

[0009] constructing a contrastive learning loss function for decentralized learning;

[0010] calculating the final local training loss function according to the degree centrality of the node;

[0011] performing one or more gradient updates on the aggregated intermediate model on the local data set according to the local training loss function;

[0012] updating the local model parameters according to the calculated gradient.

[0013] Optionally, the initialization of the local model parameters is any one or combination of independent initialization of the local model parameters by each node or local model initialization using uniform initial parameters in a preset model structure.

[0014] Optionally, the aggregation of the intermediate model according to the Gossip matrix comprises the steps of:

[0015] constructing a Gossip matrix;

[0016] Each node i performs model weighting according to the Gossip matrix to obtain an intermediate model.

[0017] Optionally, the construction of the contrastive learning loss function for decentralized learning comprises the steps of:

[0018] calculating the cosine similarity of the model representation of the neighbor node set in the current round or historical cache and the model representation of the node;

[0019] constructing a contrastive learning loss function according to the cosine similarity of the model representation with the greatest cosine similarity to the model representation of the node.

[0020] Further optionally, the calculation of the final local training loss function according to the degree centrality of the node comprises the steps of:

[0021] calculating the degree centrality of the node according to the number of neighbor nodes and the total number of nodes in the network;

[0022] calculating a structure regularization term according to the model deviation of the node and the neighbor nodes and the degree centrality of the node;

[0023] The final local training loss function is constructed according to dynamic weighting of the contrastive learning loss function and the structure regularization term.

[0024] Optionally, the final local training loss function is constructed according to dynamic weighting of the contrastive learning loss function and the structure regularization term, comprising the steps of:

[0025] The scale index of the two terms is calculated according to the numerical scale of the contrastive learning loss function and the structure regularization term.

[0026] The convergence index of the two terms is calculated according to the convergence speed difference of the contrastive learning loss function and the structure regularization term.

[0027] The volatility index of the two terms is calculated according to the data volatility of the contrastive learning loss function and the structure regularization term.

[0028] The weights of the two terms are calculated according to the scale index and / or convergence index and / or volatility index of the contrastive learning loss function and the structure regularization term.

[0029] The final local training loss function is constructed according to the contrastive learning loss function and the structure regularization term and the respective weights.

[0030] Optionally, the one or more gradient updates of the aggregated intermediate model on the local data set according to the local training loss function comprises the steps of:

[0031] The local loss gradient of the node is calculated according to the final local training loss function.

[0032] The one or more gradient updates of the aggregated intermediate model are performed according to the local loss gradient of the node and the preset learning rate.

[0033] Optionally, the local model parameters are calculated according to the calculated model gradient on the basis of the intermediate model parameters.

[0034] According to another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the above method.

[0035] According to another embodiment of the present application, a decentralized federated learning system based on structure and data perception is provided, comprising:

[0036] a processor;

[0037] a memory;

[0038] and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the signal processing unit, the programs causing the computer to perform the above method.

[0039] The structure and data perception-based decentralized federated learning device, method, system and storage medium of the present application have the advantages of:

[0040] (1) A structure perception-based regularization term design method based on node network connectivity is proposed, which introduces the topological connectivity (such as degree centrality) of nodes into the loss function, encourages the model parameters to move towards the nodes with higher propagation ability in the network, thereby suppressing the adverse effects of isolated nodes on the global model due to low data quality or training bias, and improving the convergence and robustness of the system as a whole.

[0041] (2) A data quality perception-oriented decentralized contrastive learning mechanism is constructed, which introduces a model contrastive loss into the federated learning architecture without a central server, constructs positive and negative sample pairs using neighbor nodes and historical model caches, effectively improves the consistency and discriminability of the model, and enhances the generalization performance of the model in a heterogeneous data environment.

[0042] (3) A multi-objective joint optimization objective function is designed, and an adaptive weighting mechanism based on training dynamics is introduced, which dynamically and adaptively adjusts the weighting value according to the scale index and / or convergence index and / or volatility index of the contrastive learning loss function and the structure regularization term, realizes a dynamic adjustment strategy without human intervention, and balances the convergence speed, model stability and system fairness. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a structure and data perception-based decentralized federated learning method flowchart of an embodiment of the present application.

[0044] Figure 2 is a flowchart of step S02 of the structure and data perception-based decentralized federated learning method of an embodiment of the present application.

[0045] Figure 3 is a flowchart of step S03 of the structure and data perception-based decentralized federated learning method of an embodiment of the present application.

[0046] Figure 4 is a flowchart of step S04 of the structure and data perception-based decentralized federated learning method of an embodiment of the present application.

[0047] Figure 5 is a flowchart of sub-step S043 of the structure and data perception-based decentralized federated learning method of an embodiment of the present application.

[0048] Figure 6is a flowchart of substep S05 of the decentralized federated learning method based on structure and data perception of the embodiment of the application.

[0049] Figure 7 is a structural schematic diagram of the decentralized federated learning system based on structure and data perception of the embodiment of the application. DETAILED DESCRIPTION

[0050] The application will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the application. These are within the scope of protection of the application.

[0051] The decentralized federated learning architecture adopted by the embodiment of the application is a network system composed of N nodes. Each node i has a local data set D i The topology of the network can be represented by an undirected graph showing whether there is a direct communication relationship between any two nodes. All nodes aim to collaboratively train a global model, whose parameters are denoted as w, and each node maintains its own version of local model parameters w i Without sharing the original data, the target function is continuously optimized through local training of the model and collaborative fusion of the neighbor model where represents the loss function defined on the local data set D i of node i.

[0052] According to one embodiment of the application, a decentralized federated learning method based on structure and data perception is provided, and the flowchart is as shown in Figure 1 , which comprises:

[0053] Step S01, each node initializes the local model parameters;

[0054] Step S02, in each iteration, each node collects model parameters from its neighbor node set and aggregates the intermediate model according to the Gossip matrix;

[0055] Step S03, constructing a contrastive learning loss function for decentralized learning;

[0056] Step S04, calculating the final local training loss function according to the degree centrality of the node;

[0057] Step S05, performing one or more gradient updates on the aggregated intermediate model on the local data set according to the local training loss function;

[0058] Step S06, updating the local model parameters according to the calculated gradient.

[0059] Optionally, the initializing the local model parameters is any one or combination of independently initializing the local model parameters by each node or initializing the local model with uniform initial parameters in a preset model structure. In this embodiment, each node i independently initializes the local model parameters or initializes the local model with uniform initial parameters in a preset model structure.

[0060] Optionally, the step S02 of aggregating to obtain the intermediate model according to the Gossip matrix includes the steps as shown in the flow chart Figure 2 .

[0061] Step S021 of constructing the Gossip matrix.

[0062] Step S022 of performing model weighting by each node i according to the Gossip matrix to obtain the intermediate model.

[0063] In this embodiment, in each round of iteration, each node collects model parameters from its neighbor set and performs aggregation according to the Gossip matrix. Let the Gossip matrix corresponding to the undirected graph be , the neighbors of node i are denoted as including itself, then g ij is defined as:

[0064] (1)

[0065] The matrix G satisfies the following conditions:

[0066] (1) If i≠j and node i is not connected to node j, then g ij =0; if connected, then g ij >0;

[0067] (2) G is a symmetric matrix: G=G T .

[0068] (3) G is a doubly stochastic matrix, that is, for all i, j, satisfies .

[0069] Each node i performs model weighting according to the following formula (2) to obtain the intermediate model:

[0070] (2)

[0071] Optionally, the step S03 of constructing the contrastive learning loss function for decentralized learning includes the steps as shown in the flow chart Figure 3 .

[0072] Step S031, calculate the cosine similarity between the model representation of the neighbor node set in the current round or the historical cache and the model representation of the node;

[0073] Step S032, construct a contrastive learning loss function according to the cosine similarity between the model representation with the largest cosine similarity and the model representation of the node.

[0074] In this embodiment, in order to enhance the robustness of the model to heterogeneous data distribution, the contrastive learning loss is innovatively used in the node local training process. Specifically, each node i realizes model contrastive learning through the following loss function.

[0075] Calculate the cosine similarity between the model representation of the neighbor node set in the current round or the historical cache and the model representation of the node, and construct a contrastive learning loss function according to the cosine similarity between the model representation with the largest cosine similarity and the model representation of the node. As shown in formula (3).

[0076] (3)

[0077] Where z i is the model representation of node i, is the most similar representation from the neighbors in the current round or the historical cache, is the cosine similarity, and τ is a preset temperature coefficient.

[0078] Optionally, the step S04, according to the degree centrality of the node, calculates the final local training loss function, and the flow chart is as shown in Figure 4 , including steps:

[0079] Step S041, calculate the degree centrality of the node according to the number of neighbor nodes and the total number of nodes in the network;

[0080] Step S042, calculate the structure regularization term according to the model deviation of the node and the neighbor nodes and the degree centrality of the node;

[0081] Step S043, construct the final local training loss function according to the dynamic weighting sum of the contrastive learning loss function and the structure regularization term.

[0082] In this embodiment, in order to avoid the interference of the isolated nodes with poor connectivity in the network on the model training due to data deviation, the application innovatively designs a structure-aware regularization term based on degree centrality, which is used to encourage the model to converge to the main area of the network.

[0083] Calculate the degree centrality of node i according to the number of neighbor nodes and the total number of nodes in the network , as shown in the following formula:

[0084] (4)

[0085] wherein, denotes the number of neighbors of node j, N is the total number of nodes in the network, The larger the value is, the more connected the node is, and the stronger the network propagation ability is.

[0086] On this basis, the structure regularization term is calculated according to the model deviation of the node and the neighbor node and the degree centrality of the node , which is expressed as:

[0087] (5)

[0088] Optionally, the step S043 comprises constructing a final local training loss function according to a dynamic weighting sum of the contrastive learning loss function and the structure regularization term, and a flow chart is as shown in Figure 5 , comprising the steps of:

[0089] The step S0431 comprises calculating a scale index of the contrastive learning loss function and the structure regularization term according to a numerical scale of the two items.

[0090] The step S0432 comprises calculating a convergence index of the contrastive learning loss function and the structure regularization term according to a convergence speed difference of the two items.

[0091] The step S0433 comprises calculating a volatility index of the contrastive learning loss function and the structure regularization term according to a data volatility of the two items.

[0092] The step S0434 comprises calculating a weight of the contrastive learning loss function and the structure regularization term according to the scale index and / or the convergence index and / or the volatility index of the two items.

[0093] The step S0435 comprises constructing a final local training loss function according to the contrastive learning loss function and the structure regularization term and the respective weights.

[0094] In the embodiment, since the objective function is composed of multiple loss terms, each of which is different in numerical scale, convergence speed and volatility, in order to avoid the situation that one term dominates the training and another term is ignored, the application innovatively introduces an adaptive weight coefficient setting mechanism. The historical values of the contrastive loss term and the structure regularization term of each node are obtained.

[0095] The scale index of the contrastive learning loss function and the structure regularization term is calculated according to the numerical scale of each node contrastive loss term and the structure regularization term The scale index of the two terms is calculated according to the positive correlation between the mean value of the historical value of the contrast loss term and the scale index, and the scale index of the contrast loss term is represented by variable l;

[0096] The convergence index of the two terms is calculated according to the positive correlation between the convergence speed of the historical value of the contrast loss term and the structure regular term The convergence index of the two terms is calculated according to the positive correlation between the convergence speed of the historical value of the contrast loss term and the scale index, and the scale index of the contrast loss term is represented by variable l;

[0097] The volatility index of the two terms is calculated according to the positive correlation between the variance of the historical value of the contrast loss term and the structure regular term The volatility index of the two terms is calculated according to the positive correlation between the variance of the historical value of the contrast loss term and the scale index, and the scale index of the contrast loss term is represented by variable l;

[0098] The weight of the two terms is calculated according to the positive correlation between the scale index and / or the convergence index and / or the volatility index of the contrast learning loss function and the structure regular term, and the weight value of the contrast loss term is calculated according to the positive correlation between the scale index and / or the convergence index and / or the volatility index of the contrast loss term and the weight value (normalized) of the contrast loss term, and the weight value of the structure regular term is calculated according to the weight value of the contrast loss term, so as to obtain the weight value of the two terms, and the weight value of the contrast loss term is represented by variable u.

[0099] Embodiments A1-A7 represent different implementations for calculating the weight value of the contrast loss term.

[0100] Embodiment A1: calculating the weight value according to the scale index.

[0101] Specifically, the scale index is calculated according to the positive correlation between the mean value of the historical value of the contrast loss term of each node and the scale index, and is represented by l; the weight value u is calculated according to the positive correlation between the scale index l and the weight value. In a preferred embodiment, the weight value u = o1·l o2 +o3, where o1, o2 (o2>0), o3 are calculation coefficients obtained by prior training. In this embodiment, the historical value of the node contrast loss term within a period of time is obtained, the normalized mean value is calculated as d=0.6, the scale index l = k1·d k2 +k3=1×0.6+0=0.6 (k1, k2, k3 are calculation coefficients obtained by prior training, and in this embodiment k1=1, k2=1, k3=0), the calculation coefficients o1=1, o2=1, o3=0 obtained by prior training, and the weight value u = o1·l o2+o3=1x0.6+0=0.6.

[0102] Embodiment A2: calculating the weight value according to the convergence index.

[0103] Specifically, the convergence index m is calculated according to the positive correlation between the convergence speed of each node pair comparison loss item historical value and the scale index, and the weight value u is calculated according to the positive correlation between the convergence index m and the weight value. In a preferred embodiment, the weight value u is calculated as u=o4·m o5 +o6, wherein o4, o5 (o5>0), o6 are calculation coefficients obtained by prior training. In this embodiment, the convergence speed of the node pair comparison loss item historical value in a period of time is obtained, the convergence speed is normalized as h=0.7, and the convergence index m is calculated according to the positive correlation between the convergence speed and the convergence index as m=k4·h k5 +k6=1x0.7+0=0.7 (k4, k5, k6 are calculation coefficients obtained by prior training, and in this embodiment k4=1, k5=1, k6=0), the calculation coefficients o4=1, o5=1, o6=0 obtained by prior training, and the weight value u is calculated as u=o4·m o5 +o6=1x0.7+0=0.7.

[0104] Embodiment A3: calculating the weight value according to the volatility index.

[0105] Specifically, the volatility index r is calculated according to the positive correlation between the variance of each node pair comparison loss item historical value and the scale index, and the weight value u is calculated according to the positive correlation between the volatility index r and the weight value. In a preferred embodiment, the weight value u is calculated as u=o7·r o8 +o9, wherein o7, o8 (o8>0), o9 are calculation coefficients obtained by prior training. In this embodiment, the historical value of the node pair comparison loss item in a period of time is obtained, the variance of the historical value is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r is calculated according to the positive correlation between the normalized variance and the volatility index as r=k7·a k8 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by prior training, and in this embodiment k7=1, k8=1, k9=0), the calculation coefficients o7=1, o8=1, o9=0 obtained by prior training, and the weight value u is calculated as u=o7·r o8 +o9=1x0.8+0=0.8.

[0106] Embodiment A4: calculating the weight value according to the scale index and the convergence index.

[0107] Specifically, the scale index is calculated according to the positive correlation between the mean value of the history value of each node contrast loss term and the scale index, denoted as l; the convergence index is calculated according to the positive correlation between the convergence speed of the history value of each node contrast loss term and the convergence index, denoted as m; and the weight value u is calculated according to the positive correlation between the scale index l and the convergence index m and the weight value. In a preferred embodiment, the weight value u = o10·l o11 + o12·m o13 , wherein o10, o11 (o11>0), o12, o13 (o13>0) are calculation coefficients obtained by prior training. In the embodiment, the history value of the node contrast loss term within a period of time is obtained, the normalized mean value is calculated as d = 0.6, the scale index l is calculated according to the positive correlation between the mean value and the scale index, k1·d k2 +k3 = 1×0.6 + 0 = 0.6 (k1, k2, k3 are calculation coefficients obtained by prior training; the history convergence speed of the node contrast loss term within a period of time is obtained, the convergence speed is normalized as h = 0.7, and the convergence index m is calculated according to the positive correlation between the convergence speed and the convergence index, k4·h k5 +k6 = 1×0.7 + 0 = 0.7 (k4, k5, k6 are calculation coefficients obtained by prior training, k4 = 1, k5 = 1, and k6 = 0 in the embodiment); the calculation coefficients obtained by prior training are o10 = 0.7, o11 = 1, o12 = 0.3, and o13 = 1, and the weight value u = o10·l o11 + o12·m o13 = 0.7×0.6 + 0.3×0.7 = 0.63. In another preferred embodiment, the weight value u = o14·l o15 ·m o16 + o17, wherein o14, o15 (o15>0), o16 (o16>0), and o17 are calculation coefficients obtained by prior training. In the embodiment, the history value of the node contrast loss term within a period of time is obtained, the normalized mean value is calculated as d = 0.6, the scale index l is calculated according to the positive correlation between the mean value and the scale index, k1·d k2 +k3 = 1×0.6 + 0 = 0.6 (k1, k2, k3 are calculation coefficients obtained by prior training; the history convergence speed of the node contrast loss term within a period of time is obtained, the convergence speed is normalized as h = 0.7, and the convergence index m is calculated according to the positive correlation between the convergence speed and the convergence index, k4·h k5 +k6 = 1×0.7 + 0 = 0.7 (k4, k5, k6 are calculation coefficients obtained by prior training, k4 = 1, k5 = 1, and k6 = 0 in the embodiment); the calculation coefficients obtained by prior training are o14 = 1.5, o15 = 1, o16 = 1, and o17 = 0, and the weight value u = o14·l o15 ·m o16+ o17=1.5x0.6x0.7+0=0.63.

[0108] Embodiment A5: calculating the weight value according to the scale index and the volatility index.

[0109] Specifically, the scale index is calculated according to the positive correlation between the mean value of the historical value of each node pair contrast loss term and the scale index, denoted as l; the volatility index is calculated according to the positive correlation between the variance of the historical value of each node pair contrast loss term and the scale index, denoted as r; the weight value u is calculated according to the positive correlation between the scale index l and the volatility index r and the weight value. In a preferred embodiment, the weight value u is calculated as u = o18·l o19 + o20·r o21 wherein o18, o19 (o19>0), o20, o21 (o21>0) are calculation coefficients obtained through prior training. In this embodiment, the historical value of the node pair contrast loss term within a period of time is obtained, the normalized mean value is calculated as d=0.6, and the scale index l is calculated according to the positive correlation between the mean value and the scale index, i.e. l = k1·d k2 +k3=1x0.6+0=0.6 (k1, k2, k3 are calculation coefficients obtained through prior training; the historical value of the node pair contrast loss term within a period of time is obtained, the variance of the historical value is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r is calculated according to the positive correlation between the normalized variance and the volatility index, i.e. r = k7·a k8 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained through prior training, wherein k7=1, k8=1, and k9=0 in this embodiment); the calculation coefficients o18=0.8, o19=1, o20=0.2, and o21=1 obtained through prior training, and the weight value u is calculated as u = o18·l o19 + o20·r o21 =0.8x0.6+0.2x0.8=0.64. In another preferred embodiment, the weight value u is calculated as u = o22·l o23 ·r o23 +o25, wherein o22, o23 (o23>0), o24 (o24>0), and o25 are calculation coefficients obtained through prior training. In this embodiment, the historical value of the node pair contrast loss term within a period of time is obtained, the normalized mean value is calculated as d=0.6, and the scale index l is calculated according to the positive correlation between the mean value and the scale index, i.e. l = k1·d k2 +k3=1x0.6+0=0.6 (k1, k2, k3 are calculation coefficients obtained through prior training; the historical value of the node pair contrast loss term within a period of time is obtained, the variance of the historical value is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r is calculated according to the positive correlation between the normalized variance and the volatility index, i.e. r = k7·a k8+k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by pre-training, in this embodiment, k7=1, k8=1, k9=0); the calculation coefficient o22=1.35, o23=1, o24=1, o25=0 obtained by pre-training, the calculation weight value u=o22xl o23 ·r o23 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by pre-training, in this embodiment, k7=1, k8=1, k9=0); the calculation coefficient o22=1.35, o23=1, o24=1, o25=0 obtained by pre-training, the calculation weight value u=o22xl

[0110] Embodiment A6: calculating the weight value according to the convergence index and the volatility index.

[0111] Specifically, the convergence index m is calculated according to the positive correlation between the convergence speed of each node pair comparison loss item historical value and the scale index; the volatility index r is calculated according to the positive correlation between the variance of each node pair comparison loss item historical value and the scale index; and the weight value u is calculated according to the positive correlation between the convergence index m and the volatility index r and the weight value. In a preferred embodiment, the weight value u=o26xm o27 +o28xr o29 , wherein o26, o27 (o27>0), o28, o29 (o29>0) are calculation coefficients obtained by pre-training. In this embodiment, the historical convergence speed of the node pair comparison loss item in a period of time is obtained, the convergence speed is normalized as h=0.7, and the convergence index m=k4xh is calculated according to the positive correlation between the convergence speed and the convergence index. k5 +k6=1x0.7+0=0.7 (k4, k5, k6 are calculation coefficients obtained by pre-training, in this embodiment, k4=1, k5=1, k6=0); the historical value of the node pair comparison loss item in a period of time is obtained, the variance of the historical value is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r=k7xa is calculated according to the positive correlation between the normalized variance and the volatility index. k8 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by pre-training, in this embodiment, k7=1, k8=1, k9=0); the calculation coefficient o22=1.35, o23=1, o24=1, o25=0 obtained by pre-training, the calculation weight value u=o22xl o27 +o28xr o29 =0.7x0.7+0.3x0.8=0.73. In another preferred embodiment, the weight value u=o30xm o31 xr o32+o33, wherein o30, o31 (o31 > 0), o32 (o32 > 0), o33 are calculation coefficients obtained by prior training. In the embodiment, the historical convergence speed of the node contrast loss item in a period of time is obtained, the convergence speed is normalized as h = 0.7, and the convergence index m = k4 h is calculated according to the positive correlation between the convergence speed and the convergence index. k5 +k6 = 1 x 0.7 + 0 = 0.7 (k4, k5, k6 are calculation coefficients obtained by prior training; in the embodiment, k4 = 1, k5 = 1, and k6 = 0); the historical value of the node contrast loss item in a period of time is obtained, the variance of the historical value is calculated and normalized to obtain the normalized variance a = 0.8, and the volatility index r = k7 a is calculated according to the positive correlation between the normalized variance and the volatility index. k8 +k9 = 1 x 0.8 + 0 = 0.8 (k7, k8, k9 are calculation coefficients obtained by prior training; in the embodiment, k7 = 1, k8 = 1, and k9 = 0); the calculation coefficients o30 = 1.2, o31 = 1, o32 = 1, and o33 = 0 obtained by prior training are calculated, and the weight value u = o30 m is calculated. o31 ·r o32 +o33 = 1.2 x 0.7 x 0.8 + 0 = 0.672.

[0112] Embodiment A7: calculating the weight value according to the scale index, the convergence index, and the volatility index.

[0113] Specifically, the scale index is calculated according to the positive correlation between the mean value of the historical value of each node contrast loss item and the scale index, which is represented as l; the convergence index is calculated according to the positive correlation between the convergence speed of the historical value of each node contrast loss item and the scale index, which is represented as m; the volatility index is calculated according to the positive correlation between the variance of the historical value of each node contrast loss item and the scale index, which is represented as r; and the weight value u is calculated according to the positive correlation between the scale index l, the convergence index m, the volatility index r, and the weight value. In a preferred embodiment, the weight value u = o34 l is calculated. o35 +o36 m o37 +o38 r o39 wherein o34, o35 (o35 > 0), o36, o37 (o37 > 0), o38, o39 (o39 > 0) are calculation coefficients obtained by prior training. In the embodiment, the historical value of the node contrast loss item in a period of time is obtained, the normalized mean value d = 0.6 is calculated, and the scale index l = k1 d is calculated according to the positive correlation between the mean value and the scale index. k2 +k3 = 1 x 0.6 + 0 = 0.6 (k1, k2, k3 are calculation coefficients obtained by prior training; the historical convergence speed of the node contrast loss item in a period of time is obtained, the convergence speed is normalized as h = 0.7, and the convergence index m = k4 h is calculated according to the positive correlation between the convergence speed and the convergence index.k5 +k6=1x0.7+0=0.7 (k4, k5, k6 are calculation coefficients obtained by prior training, k4=1, k5=1, k6=0 in this embodiment); the historical value of the node contrast loss term in a period of time is obtained, the variance of the history is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r=k7-a is calculated according to the positive correlation relationship thereof with the volatility index k8 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by prior training, k7=1, k8=1, k9=0 in this embodiment); the calculation coefficients o34=0.5, o35=1, o26=0.3, o27=1, o28=0.2, o29=1 obtained by prior training, the weight value u=o34-l o35 +o36-m o37 +o38-r o39 =0.5x0.6+0.3x0.7+0.2x0.8=0.67. In another preferred embodiment, the weight value u=o40-l o41 -m o42 -r o43 +o44, wherein o40, o41 (o41>0), o42 (o42>0), o43 (o43>0), o44 are calculation coefficients obtained by prior training. In this embodiment, the historical value of the node contrast loss term in a period of time is obtained, the normalized mean value d=0.6 is calculated, and the scale index l=k1-d is calculated according to the positive correlation relationship thereof with the scale index k2 +k3=1x0.6+0=0.6 (k1, k2, k3 are calculation coefficients obtained by prior training; the historical convergence speed of the node contrast loss term in a period of time is obtained, the convergence speed is normalized to h=0.7, and the convergence index m=k4-h is calculated according to the positive correlation relationship thereof with the convergence index k5 +k6=1x0.7+0=0.7 (k4, k5, k6 are calculation coefficients obtained by prior training, k4=1, k5=1, k6=0 in this embodiment); the historical value of the node contrast loss term in a period of time is obtained, the variance of the history is calculated and normalized to obtain the normalized variance a=0.8, and the volatility index r=k7-a is calculated according to the positive correlation relationship thereof with the volatility index k8 +k9=1x0.8+0=0.8 (k7, k8, k9 are calculation coefficients obtained by prior training, k7=1, k8=1, k9=0 in this embodiment); the calculation coefficients o40=2, o41=1, o42=1, o43=1, o44=0 obtained by prior training, the weight value u=o40-l o41 -m o42 -r o43+o44=2x0.6x0.7x0.8+0=0.672.

[0114] According to the method of any one of embodiments A1-A7, the weight value u of the contrast loss term is calculated, and the weight value v of the structure regularization term is 1-u.

[0115] According to the contrast learning loss function and the structure regularization term and respective weights, a final local training loss function L is constructed. i , which is expressed as:

[0116] (6)

[0117] As an option, minimum and maximum weight values are set to avoid complete neglect of a certain loss function, for example, let .

[0118] Optionally, the step S05 performs one or more gradient updates on the aggregated intermediate model on the local data set according to the local training loss function, and a flowchart is shown as Figure 6 , which includes the steps of:

[0119] Step S051 calculates the local loss gradient of the node according to the final local training loss function.

[0120] Step S052 performs one or more gradient updates on the aggregated intermediate model according to the local loss gradient of the node and a preset learning rate.

[0121] In this embodiment, the local loss gradient of the node is calculated according to the final local training loss function L i , which is expressed as , and each node performs one or more gradient updates on the intermediate model on the local data set, which is expressed as:

[0122] (7)

[0123] wherein is a preset learning rate.

[0124] Optionally, the step S06 updates the local model parameters according to the calculated gradient, which is that each node calculates the local model parameters according to the calculated model gradient on the basis of the intermediate model parameters, which is expressed as:

[0125] (8)

[0126] According to another embodiment of the present application, a computer readable storage medium is also provided, which stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the structure and data perception based decentralized federated learning method described in the above embodiments.

[0127] According to another embodiment of the present invention, a decentralized federated learning system based on structure and data awareness is also provided, the structural diagram of which is shown below. Figure 7 As shown, it includes:

[0128] processor;

[0129] Memory;

[0130] And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by the processor, the programs causing the computer to perform the structure- and data-aware decentralized federated learning method described in the above embodiments.

[0131] The methods described above according to the invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an AuIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the processes shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for performing the processes shown herein.

[0132] Of course, those skilled in the art should recognize that the above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Any changes or modifications to the above embodiments that are within the scope of the present invention will fall within the protection scope of the present invention.

Claims

1. A method for structure and data-aware decentralized federated learning, the method comprising: The method comprises the following steps: each node initializes local model parameters; in each iteration, each node collects model parameters from its neighbor node set and aggregates the intermediate model according to the Gossip matrix; constructing a contrastive learning loss function for decentralized learning; calculating the final local training loss function according to the degree centrality of the node; performing one or more gradient updates on the aggregated intermediate model on the local data set according to the local training loss function; updating the local model parameters according to the calculated gradient.

2. The method of claim 1, wherein, The initialization of the local model parameters is any one or combination of the following: each node independently initializes the local model parameters or uses the uniform initial parameters in the preset model structure to initialize the local model.

3. The method of claim 1, wherein, The aggregation of the intermediate model according to the Gossip matrix comprises the following steps: constructing a Gossip matrix; each node i performs model weighting according to the Gossip matrix to obtain an intermediate model.

4. The method of claim 1, wherein, The construction of the contrastive learning loss function for decentralized learning comprises the following steps: calculating the cosine similarity of the model representation of the neighbor node set in the current round or the history cache and the model representation of the node; constructing a contrastive learning loss function according to the cosine similarity of the model representation with the largest cosine similarity with the model representation of the node.

5. The method of claim 1, wherein, The calculation of the final local training loss function according to the degree centrality of the node comprises the following steps: calculating the degree centrality of the node according to the number of neighbor nodes and the total number of nodes in the network; calculating the structure regularization term according to the model deviation of the node and the neighbor nodes and the degree centrality of the node; constructing the final local training loss function according to the dynamic weighting sum of the contrastive learning loss function and the structure regularization term.

6. The method of claim 5, wherein the method further comprises: The construction of the final local training loss function according to the dynamic weighting sum of the contrastive learning loss function and the structure regularization term comprises the following steps: calculating the scale index of the two items according to the numerical scale of the contrastive learning loss function and the structure regularization term; calculating the convergence index of the two items according to the convergence speed difference of the contrastive learning loss function and the structure regularization term; calculating the volatility index of the two items according to the data volatility of the contrastive learning loss function and the structure regularization term; calculating the weight of the two items according to the scale index and / or convergence index and / or volatility index of the contrastive learning loss function and the structure regularization term; constructing the final local training loss function according to the contrastive learning loss function and the structure regularization term and their respective weights.

7. The method of claim 1, wherein, The one or more gradient updates on the aggregated intermediate model on the local data set according to the local training loss function comprises the following steps: calculating the local loss gradient of the node according to the final local training loss function; performing one or more gradient updates on the aggregated intermediate model according to the local loss gradient of the node and the preset learning rate.

8. The method of claim 1, wherein, The updating of the local model parameters according to the calculated gradient is that each node calculates the local model parameters according to the calculated model gradient based on the intermediate model parameters.

9. A computer readable storage medium storing a computer program for electronic data interchange, wherein, The computer program enables the computer to perform the method of any one of claims 1-8. 10.A structure and data perception based decentralized federated learning system, characterized in that, The method comprises the following steps: a processor; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs causing the computer to perform the method of any one of claims 1-8.

Images