Decentralized federated learning byzantine robust consensus method

Abstract

The application discloses a Byzantine robust consensus method for decentralized federated learning and relates to the technical field of distributed machine learning security. The method comprises the following steps: a processing terminal acquires a current local model, local model update data and a neighbor model update message set, generates an update direction signature based on a neighbor model update vector and divides a direction similar update group; in combination with a propagation source digest, a propagation hop count digest and arrival time information, a propagation source overlap degree, an arrival time sequence chain consistency and a multi-hop echo amplification index are generated; based on the multi-hop echo amplification index, original support weights are decorrelated and compressed to obtain effective independent support weights, and a local consensus update is generated based on the effective independent support weights to update the current local model. The method can inhibit the repeated weighting of homologous updates in multi-hop propagation, and improve the consensus robustness of decentralized federated learning in a Byzantine environment.

Description

Technical Field

[0001] This invention relates to the field of distributed machine learning security technology, and more specifically, this application relates to a decentralized federated learning Byzantine robust consensus method. Background Technology

[0002] With the increasing application of federated learning in edge computing, the Internet of Things, and distributed intelligent systems, training models collaboratively by multiple terminals without uploading raw data has become a common technical approach. Compared to federated learning that relies on a central server, decentralized federated learning typically involves each node exchanging model parameter changes with its neighboring nodes and performing aggregation and updates locally. This reduces the risks associated with central node failures or attacks.

[0003] Existing solutions, when dealing with Byzantine nodes, typically filter out updates that deviate significantly from the majority direction based on the distance, similarity, or historical reliability of model updates sent by neighboring nodes, or enhance the influence of consistent updates from multiple directions in local aggregation. The implicit premise of this type of solution is that when multiple neighboring nodes send model updates in similar directions, these updates can be considered as multiple independent pieces of supporting evidence.

[0004] However, in decentralized networks, model updates are not static data that arrive all at once. Instead, they are absorbed, forwarded, and re-aggregated through multiple rounds of communication between nodes. Thus, the same upstream bias may first affect some nodes, and then propagate through these nodes to other neighboring nodes, ultimately causing the target node to receive multiple updates in similar directions in the same or adjacent training rounds. At this point, these updates, seemingly from different neighboring nodes, may actually carry the repeated effects of the same propagation bias.

[0005] If the system continues to directly accumulate updates with similar directions as multiple independent pieces of evidence, the bias that should have been absorbed to a limited extent will be amplified due to repeated weighting. Especially when malicious nodes use low-amplitude, continuous propagation methods, a single update may not be obviously abnormal, but multiple similar updates amplified by the propagation chain will gradually influence the local aggregation direction, thus affecting the model's convergence stability and robustness. Therefore, the problem that existing technologies need to solve is: in the normal propagation process of decentralized federated learning, how to avoid the bias of homologous propagation being repeatedly represented by multiple neighbor updates and still participating in consensus aggregation as independent supporting evidence. Summary of the Invention

[0006] To address the aforementioned technical issues, this technical solution provides a decentralized federated learning Byzantine robust consensus method, resolving the problems raised in the background section.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] This application provides a decentralized federated learning Byzantine robust consensus method for robust training of machine learning models in decentralized federated learning, executed by a processing terminal, including:

[0009] Get the current local model, local model update data, and neighbor model update message set composed of neighbor model update messages for the current training round. The current local model is the local machine learning model, and the neighbor model update message includes the neighbor model update vector, propagation source summary, propagation hop count summary, and arrival time information.

[0010] Based on the neighbor model update vector, an update direction signature representing its directional attribute is generated, and based on the directional similarity between update direction signatures, neighbor model update messages that meet the preset directional similarity conditions are divided into directional similarity update groups.

[0011] Based on the direction similarity, the update direction signature of each neighbor model update message in the group is updated to generate the direction similarity within the group. Based on the corresponding propagation source summary, the propagation source overlap is generated. Based on the corresponding propagation hop count summary and arrival time information, the arrival time sequence consistency is generated.

[0012] Based on intra-group directional similarity, propagation source overlap, arrival time chain consistency, and the number of messages within the directional similarity update group, a multi-hop echo amplification index is generated to represent the degree of non-independent support of neighbor model update messages within the directional similarity update group.

[0013] Based on the multi-hop echo amplification index, decorrelation compression is performed on the original support weights of the directional similarity update group to obtain effective independent support weights;

[0014] Robust aggregation of neighbor model update vectors is performed based on effective independent support weights, and robust aggregation is also performed in conjunction with local model update data to obtain local consensus updates. The current local model is then updated based on the local consensus updates.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] This application generates update direction signatures based on the update vector of the neighbor model and divides the update groups into directionally similar groups according to the direction similarity, which reduces the impact of update magnitude differences on the comparison of neighbor model updates and provides a stable grouping basis for subsequent identification of homologous propagation updates.

[0017] This application generates propagation source overlap, arrival time chain consistency, and multi-hop echo amplification index by combining propagation source summary, propagation hop count summary, and arrival time information. This quantifies the degree of non-independent support of neighbor model update messages within the directional similarity update group, and avoids directly treating multiple directional similar updates as multiple independent supporting evidence.

[0018] This application uses the multi-hop echo amplification index as a non-independence compression factor to decorrelate and compress the original support weights, thereby obtaining effective independent support weights. This prevents repeated updates formed by homologous propagation from being accumulated based on multiple independent pieces of evidence, thus improving the consensus stability of decentralized federated learning in the Byzantine environment. Attached Figure Description

[0019] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0020] Figure 1 This is a flowchart of the decentralized federated learning Byzantine robust consensus method in this invention;

[0021] Figure 2 This is a data flow diagram of the decentralized federated learning Byzantine robust consensus method in this invention. Detailed Implementation

[0022] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0023] In decentralized federated learning scenarios, multiple processing terminals typically do not rely on a central server, but instead complete distributed training through the exchange of model updates between neighboring nodes. Each processing terminal needs to receive model updates from its neighboring nodes in each training round and combine this with its own local training results to generate a consensus update for updating its current local model. Therefore, in the absence of a central trusted aggregation node, identifying a reliable consensus direction from neighboring model updates is a key issue affecting the stability and security of decentralized federated learning.

[0024] Existing Byzantine robust aggregation methods typically assume that model updates sent by different neighboring nodes are independent of each other; when multiple neighboring model updates are in similar directions, the system often treats them as joint support for the same update direction from multiple nodes. However, in decentralized federated learning, model updates are received, absorbed, forwarded, and re-aggregated among nodes in multiple rounds. The same upstream deviation may continue to propagate through multiple neighboring nodes, causing the target processing terminal to receive multiple model updates in similar directions but with non-independent sources.

[0025] For example, in a decentralized training network composed of edge terminals, a node generates a model update with a directional offset. This offset is first absorbed by neighboring nodes and then propagated along with the model updates of neighboring nodes. When the target processing terminal receives multiple neighbor model updates with similar directions in the same training round, conventional methods can only identify that their directions are consistent, but it is difficult to determine whether these updates are formed by the same upstream offset through multi-hop propagation. Since the magnitude of a single offset update may be small, methods based on abnormal magnitude, distance thresholds, or single-node confidence are also difficult to effectively identify.

[0026] If the aforementioned problems are not addressed, the processing terminal will mistakenly accumulate updates from multiple directionally similar models formed by homologous propagation as multiple independent supporting evidence. This causes upstream bias to be repeatedly weighted during the local consensus process, and continues to propagate and amplify in subsequent training rounds, forming a pseudo-majority consensus. Consequently, this reduces the model's convergence stability and consensus robustness. Therefore, a decentralized federated learning Byzantine robust consensus method is needed that can identify the degree of non-independent support in updates from directionally similar models and suppress the repeated weighting of homologous propagation bias.

[0027] like Figure 1-2 As shown, this application introduces a decentralized federated learning Byzantine robust consensus method, executed by a processing terminal, including:

[0028] S1. Obtain the current local model, local model update data, and a set of neighbor model update messages composed of neighbor model update messages for the current training round. The neighbor model update messages include the neighbor model update vector, propagation source summary, propagation hop count summary, and arrival time information.

[0029] Specifically, at the start of the current training round, the processing terminal obtains the current local model stored locally, which is a local machine learning model; the processing terminal performs local training on the current local model based on the local training data to obtain local model update data, which is used to characterize the parameter changes of the current local model before and after the current training round.

[0030] The current local model is the machine learning model to be updated that is stored locally on the processing terminal before the start of the current training round. The current local model serves as the initial model state for local training and aggregation of updates from neighboring models in the current training round. The current local model can be a neural network model, a classification model, a regression model, or other machine learning model whose parameters can be updated using training data.

[0031] The processing terminal receives neighbor model update messages within the neighbor message reception time window of the current training round, and extracts the neighbor model update vector, propagation source summary, propagation hop count summary, and arrival time information from the neighbor model update messages. Specifically, the neighbor model update vector represents the amount of model parameter change formed by neighbor nodes in the current training round; the propagation source summary represents the upstream update source absorbed by the neighbor model update message; the propagation hop count summary represents the number of propagation hops corresponding to the neighbor model update message; and the arrival time information represents the time when the processing terminal receives the neighbor model update message.

[0032] The processing terminal determines whether the neighbor model update message contains the neighbor model update vector, propagation source digest, propagation hop count digest, and arrival time information, and determines whether the dimension of the neighbor model update vector is consistent with the parameter dimension of the current local model; if they are consistent, the neighbor model update message is written to the neighbor model update message set; otherwise, it is not written to the neighbor model update message set.

[0033] After the above processing, the processing terminal outputs the current local model, local model update data and neighbor model update message set for the current training round, which serve as the data basis for subsequent generation of update direction signatures, division of directional similarity update groups and generation of multi-hop echo amplification index.

[0034] Example: For instance, the processing terminal reads the current local model in the 35th training round. The current local model has 120,000 parameters. The processing terminal performs one round of local training on the current local model based on the local training data, obtaining updated local model data with 120,000 dimensions.

[0035] Within the neighbor message reception time window of the 35th training round, the processing terminal received four neighbor model update messages. The neighbor model update messages of neighbor nodes A, B, and C all included a 120,000-dimensional neighbor model update vector, a 256-bit propagation source summary, a propagation hop count summary, and arrival time information; the neighbor model update message of neighbor node D lacked a propagation source summary.

[0036] The processing terminal writes the neighbor model update messages of neighbor nodes A, B, and C into the neighbor model update message set, but does not write the neighbor model update message of neighbor node D into the neighbor model update message set. Finally, the processing terminal outputs the current local model, local model update data, and the neighbor model update message set containing the three neighbor model update messages of neighbor nodes A, B, and C for the 35th training round.

[0037] Through the above technical solution, this application solves the problems of unclear sources and inconsistent data structures between local update data and neighbor update messages in decentralized federated learning. By synchronously acquiring the current local model, local model update data and neighbor model update message set in the current training round, and confirming the integrity and dimensional consistency of neighbor model update vector, propagation source summary, propagation hop count summary and arrival time information, it ensures that subsequent update direction signature generation, direction similarity update group division and multi-hop echo amplification index calculation all have a unified data foundation.

[0038] S2. Generate update direction signatures representing the directional attributes based on the update vectors of neighbor models, and divide neighbor model update messages that meet the preset directional similarity conditions into directional similarity update groups based on the directional similarity between update direction signatures.

[0039] The process of obtaining the directional similarity update group includes:

[0040] Obtain the update direction signature of each neighbor model update message in the neighbor model update message set;

[0041] The inner product between the update direction signatures corresponding to the update messages of any two neighbor models is calculated by vector inner product operation;

[0042] When the inner product result is not less than zero, the inner product result is determined as the directional similarity between the update messages of the corresponding two neighbor models;

[0043] When the inner product result is less than zero, the directional similarity between the update messages of the corresponding two neighbor models is set to zero.

[0044] Establish a directional association between update messages of two neighboring models whose directional similarity reaches a preset directional similarity threshold;

[0045] For example, the preset directional similarity threshold is 0.90; when the directional similarity between two neighbor model update messages reaches 0.90, the processing terminal establishes a directional association between them; when the directional similarity does not reach 0.90, the processing terminal does not establish a directional association between them.

[0046] Based on directional association, connected subsets are determined in the neighbor model update message set, and these connected subsets are used as directional similarity update groups.

[0047] Among them, neighbor model update messages in the same direction similar update group are directly or indirectly related through directional association.

[0048] Specifically, the processing terminal obtains the neighbor model update vector of each neighbor model update message in the neighbor model update message set, and performs normalization processing on the neighbor model update vector to obtain the update direction signature used to characterize the directional attribute of the neighbor model update vector.

[0049] The processing terminal performs a vector inner product operation based on the update direction signatures of any two neighbor model update messages to obtain the inner product result. When the inner product result is not less than zero, the inner product result is used as the directional similarity between the corresponding two neighbor model update messages. When the inner product result is less than zero, the directional similarity between the corresponding two neighbor model update messages is determined to be zero.

[0050] The processing terminal establishes a directional association between two neighbor model update messages whose directional similarity reaches a preset directional similarity threshold, and determines a connected subset in the set of neighbor model update messages based on the directional association, and uses the connected subset as a directional similarity update group.

[0051] After the above processing, the processing terminal outputs the update direction signature and direction similarity update group of each neighbor model update message, which serves as the basis for subsequent generation of intra-group direction similarity, propagation source overlap and arrival time sequence consistency.

[0052] Example: For instance, in the 35th training round, the processing terminal obtains three neighbor model update messages from neighbor nodes A, B, and C. The processing terminal generates update direction signatures based on the 120,000-dimensional neighbor model update vectors in each of the three neighbor model update messages.

[0053] The processing terminal calculates the directional similarity between neighboring nodes A and B to be 0.94, between neighboring nodes A and C to be 0.42, and between neighboring nodes B and C to be 0.38. If the preset directional similarity threshold is 0.90, the processing terminal establishes a directional association between neighboring nodes A and B, and divides the neighbor model update messages corresponding to neighboring nodes A and B into a first directional similarity update group, and divides the neighbor model update messages corresponding to neighboring node C into a second directional similarity update group.

[0054] Through the above technical solution, this application solves the problem that direct comparison based on neighbor model update vectors is easily affected by the difference in update amplitude. By converting the neighbor model update vector into an update direction signature representing the directional attribute, and establishing a directional association relationship based on the directional similarity between update direction signatures, neighbor model update messages with the same direction are divided into directional similar update groups, thereby providing a grouping basis for subsequent identification of whether multiple directional similar updates belong to the same source propagation echo.

[0055] S3. Based on the direction similarity, update the update direction signature of each neighbor model update message in the group to generate the direction similarity in the group, generate the propagation source overlap based on the corresponding propagation source summary, and generate the arrival time sequence consistency based on the corresponding propagation hop count summary and arrival time information.

[0056] The process of generating the overlap of propagation sources includes:

[0057] Obtain a summary of the propagation sources of update messages from neighboring models within the direction similarity update group;

[0058] Calculate the size of the intersection and union of the summaries of any two propagation source summaries;

[0059] Based on the ratio between the size of the intersection of the summaries and the size of the union of the summaries, the source overlap between the update messages of the corresponding two neighbor models is obtained;

[0060] The average overlap of each source within the directional similarity update group is processed to obtain the propagation source overlap of the directional similarity update group.

[0061] Among them, the propagation source digest is the digest data obtained by compressing and encoding the upstream update source absorbed by the neighbor model update message.

[0062] The process of generating consistent arrival time-series chains includes:

[0063] Obtain the propagation hop count summary and arrival time information of each neighbor model update message in the direction similarity update group;

[0064] Based on the historical communication records of the processing terminal, generate the normal propagation delay and propagation delay fluctuation scale corresponding to the number of propagation hops per unit;

[0065] Arrival time difference is generated based on the absolute value of the difference between the arrival time information of the update messages of any two neighbor models, and propagation hop difference is generated based on the absolute value of the difference between the propagation hop count represented by the propagation hop count digest of the update messages of any two neighbor models.

[0066] Based on the arrival time difference, propagation hop difference, regular propagation delay, and propagation delay fluctuation scale, the temporal chain consistency value between the update messages of the corresponding two neighbor models is obtained;

[0067] Timing chain consistency value satisfy: ;

[0068] in, , This indicates the arrival time information of the neighbor model update messages i and j. , This represents the corresponding propagation hop count summary. Indicates the normal propagation delay. Indicates the scale of propagation delay fluctuation. This represents a preset minimum positive number; for example, when the time unit is seconds, the preset minimum positive number is taken as... This preset minimum positive number is only used to ensure computational stability.

[0069] The consistency values ​​of each time-series chain within the direction-similar update group are averaged to obtain the consistency of the arrival time-series chain of the direction-similar update group.

[0070] The process of generating the conventional propagation delay and propagation delay fluctuation scale corresponding to a unit propagation hop count based on the historical communication records of the processing terminal includes:

[0071] The historical neighbor model update message received by the processing terminal in the historical training round is obtained. The historical neighbor model update message includes historical arrival time information and historical propagation hop count summary.

[0072] The historical arrival time difference is generated based on the absolute value of the difference between the historical arrival time information of messages updated by any two historical neighbor models.

[0073] The historical propagation hop difference is generated based on the absolute value of the difference between the propagation hops represented by the historical propagation hop count summary of any two historical neighbor models update messages;

[0074] When the historical hop count difference is greater than zero, divide the historical arrival time difference by the historical hop count difference to obtain the unit hop count delay sample.

[0075] The mean value is calculated for multiple unit hop delay samples to obtain the normal propagation delay corresponding to a unit propagation hop count;

[0076] A propagation delay fluctuation scale is generated based on the deviation of multiple unit hop delay samples relative to the conventional propagation delay.

[0077] Specifically, the processing terminal obtains the update direction signature, propagation source digest, propagation hop count digest, and arrival time information of the update messages of each neighbor model in the direction similarity update group.

[0078] The processing terminal generates intra-group directional similarity based on the directional similarity between the signatures of each update direction within the directional similarity update group; it generates source overlap between corresponding two neighbor model update messages based on the ratio of the size of the intersection of the digests and the size of the union of the digests between any two propagation source digests within the directional similarity update group, and performs averaging on the intra-group source overlap to obtain the propagation source overlap.

[0079] The processing terminal generates the normal propagation delay and propagation delay fluctuation scale corresponding to the unit propagation hop count based on historical communication records; then it generates the arrival time difference based on the absolute value of the difference between the arrival time information of any two neighbor model update messages, generates the propagation hop count difference based on the absolute value of the difference between the propagation hop counts represented by the propagation hop count summary, and generates the timing chain consistency value based on the arrival time difference, propagation hop count difference, normal propagation delay and propagation delay fluctuation scale. The timing chain consistency value within the group is averaged to obtain the arrival timing chain consistency.

[0080] After the above processing, the processing terminal outputs the intra-group directional similarity, propagation source overlap, and arrival time sequence consistency of each directional similarity update group, which serve as the data basis for the subsequent generation of the multi-hop echo amplification index.

[0081] Example: For instance, the first directional similarity update group includes neighboring nodes A and B. The processing terminal has obtained a directional similarity of 0.94 between A and B, therefore the intra-group directional similarity of the first directional similarity update group is determined to be 0.94.

[0082] The processing terminal compares the 256-bit propagation source summaries of A and B, and calculates that the summary intersection size is 5 and the summary union size is 6. Therefore, the source overlap between A and B is 5 / 6, or 0.83. The processing terminal determines the propagation source overlap of the first direction similarity update group to be 0.83.

[0083] The processing terminal obtains that A has a propagation hop count of 1 and an arrival time of 10:00:01:120; B has a propagation hop count of 2 and an arrival time of 10:00:01:310. Based on historical communication records, the processing terminal determines that the normal propagation delay corresponding to a unit propagation hop count is 0.20 seconds, and the propagation delay fluctuation scale is 0.05 seconds. The arrival time difference between A and B is 0.190 seconds, and the propagation hop count difference is 1. The timing chain consistency value corresponding to the two is relatively high. The processing terminal determines the arrival timing chain consistency of the first direction similarity update group to be 0.82.

[0084] Through the above technical solution, this application solves the problem that the existing technology can only judge consensus support based on the similarity of update direction and cannot distinguish between independent support and homologous propagation. By generating intra-group directional similarity, propagation source overlap and arrival time sequence consistency within the directional similarity update group, the three dimensions of update direction, upstream source and propagation time sequence are correlated and analyzed, thereby providing calculable data basis for judging whether directional similarity update constitutes multi-hop echo.

[0085] S4. Based on the intra-group directional similarity, propagation source overlap, arrival time chain consistency, and the number of messages within the directional similarity update group, generate a multi-hop echo amplification index that represents the degree of non-independent support of neighbor model update messages within the directional similarity update group.

[0086] The process of generating the multi-hop echo amplification index includes:

[0087] Get the intra-group directional similarity, propagation source overlap, arrival time chain consistency, and message count of the directional similarity update group;

[0088] The echo correlation strength is obtained by performing multiplicative coupling processing on intragroup directional similarity, propagation source overlap, and arrival time sequence consistency.

[0089] A multi-hop echo amplification index is generated based on echo correlation strength and message quantity;

[0090] Multi-jump echo amplification index satisfy: ;

[0091] in, Indicates the number of messages. Indicates intra-group directional similarity. Indicates the degree of overlap in the sources of transmission. This indicates consistency of the arrival time chain;

[0092] Among them, echo correlation strength is used to characterize the degree to which neighbor model update messages in the same direction similarity update group meet the common requirements in three dimensions: direction consistency, source overlap, and temporal chain matching.

[0093] Specifically, the processing terminal obtains the intra-group directional similarity of the directional update group, the overlap of propagation sources, the consistency of arrival time chain, and the number of messages.

[0094] The processing terminal performs multiplicative coupling processing on the intra-group directional similarity, propagation source overlap, and arrival time sequence consistency to obtain the echo association strength. Then, based on the echo association strength and the number of messages within the directional similarity update group, a multi-hop echo amplification index is generated. The multi-hop echo amplification index characterizes the degree of non-independent support of neighbor model update messages within the directional similarity update group.

[0095] After the above processing, the processing terminal outputs the multi-hop echo amplification index of the similar update group in each direction, which serves as the data basis for subsequent decorrelation compression of the original support weights.

[0096] Example: For instance, the intra-group directional similarity of the first directional similarity update group is 0.94, the propagation source overlap is 0.83, the arrival time chain consistency is 0.82, and the number of messages is 2. The processing terminal performs multiplicative coupling on these three data items, resulting in an echo correlation strength of approximately 0.64; combined with the message count of 2, the multi-hop echo amplification index of the first directional similarity update group is 1.64.

[0097] The second direction similarity update group includes only one neighbor model update message from neighbor node C. The processing terminal sets its multi-hop echo amplification index to 1, which indicates that there is no situation in which multiple messages in the group are repeated as independent supporting evidence.

[0098] Through the above technical solution, this application solves the problem that update messages of multiple directional similar neighbor models are directly accumulated as multiple independent supporting evidence. By using intra-group directional similarity, propagation source overlap, arrival time chain consistency and message quantity together to generate a multi-hop echo amplification index, the non-independent support degree of intra-group neighbor model update messages of directional similarity updates is quantified, thereby providing a basis for subsequent decorrelation compression of the original support weights.

[0099] S5. Based on the multi-hop echo amplification index, the original support weights of the direction similarity update group are subjected to decorrelation compression to obtain effective independent support weights;

[0100] The process of obtaining effective independent support weights includes:

[0101] Obtain the basic support weights for the model update messages of each neighbor model within the direction similarity update group;

[0102] Sum the basic support weights within the directional similarity update group to obtain the original support weights of the directional similarity update group.

[0103] The multi-hop echo amplification index is used as a non-independent compression factor for the directional similarity update group to compress the original support weights, resulting in effective independent support weights.

[0104] Effective independent support weights satisfy: ;

[0105] in, Indicates a direction similarity update group. Indicates the basic support weight. This indicates the multi-hop echo amplification index.

[0106] Specifically, the processing terminal obtains the basic support weights of each neighbor model update message within the directional similarity update group, and performs summation processing on each basic support weight within the same directional similarity update group to obtain the original support weights of that directional similarity update group.

[0107] The processing terminal uses the multi-hop echo amplification index of the similarity update group in that direction as a non-independence compression factor to compress the original support weights, thus obtaining effective independent support weights. The larger the multi-hop echo amplification index, the higher the degree of compression of the original support weights; when the multi-hop echo amplification index is 1, the original support weights are not compressed due to multi-hop echoes.

[0108] After the above processing, the processing terminal outputs the effective independent support weights of similar update groups in each direction, which serve as the data basis for subsequent generation of intra-group neighbor consensus updates, neighbor-side consensus updates, and local consensus updates.

[0109] Example: For instance, a first-direction similarity update group includes neighboring nodes A and B. A's base support weight is 1.00, and B's base support weight is 1.00. Therefore, the original support weight of the first-direction similarity update group is 2.00. Since the multi-hop echo amplification index of the first-direction similarity update group is 1.64, the processing terminal performs compression on the original support weight of 2.00, resulting in an effective independent support weight of approximately 1.22.

[0110] The second direction similarity update group includes neighbor node C, whose basic support weight is 1.00. The multi-hop echo amplification index of the second direction similarity update group is 1, so its effective independent support weight is 1.00.

[0111] Through the above technical solution, this application solves the problem of pseudo-majority support in robust aggregation due to the large number of messages in directional similarity update groups. By using the multi-hop echo amplification index as a non-independence compression factor, decorrelation compression is performed on the original support weights of directional similarity update groups to obtain effective independent support weights. This prevents repeated updates formed by homologous propagation from accumulating as multiple independent evidences, thereby improving the reliability of support weights when neighbor model updates participate in aggregation.

[0112] S6. Robustly aggregate the neighbor model update vectors according to the effective independent support weights to obtain the local consensus update, and combine it with the local model update data to obtain the local consensus update. Update the current local model based on the local consensus update.

[0113] The process of obtaining a local consensus update includes:

[0114] Obtain the effective independent support weights of each directional similarity update group, the neighbor model update vectors within each directional similarity update group, and the local model update data for the current training round;

[0115] Based on the effective independent support weights, the neighbor model update vectors in each directional similarity update group are weighted within the group to obtain the intra-group neighbor consensus update corresponding to each directional similarity update group.

[0116] Based on the effective independent support weights of similar update groups in each direction, inter-group weighted processing is performed on the neighbor consensus update within each group to obtain the neighbor-side consensus update.

[0117] Calculate the directional consistency between the local model update data and the neighbor consensus update, generate local reference weights and neighbor receiving weights based on the directional consistency, and perform weighted synthesis on the local model update data and the neighbor consensus update according to the local reference weights and the neighbor receiving weights to obtain the local consensus update;

[0118] Consistency of direction satisfy: ;

[0119] in, This indicates the local model update data in the current training round ttt. This indicates the neighbor-side consensus update in the current training round ttt. This represents the L2 norm operation, used to calculate the length of a vector;

[0120] The local model is updated based on local consensus updates to obtain the local model update result.

[0121] Specifically, the processing terminal obtains the effective independent support weights of similarity update groups in each direction, the neighbor model update vectors within each similarity update group, and the local model update data for the current training round.

[0122] The processing terminal performs intra-group weighted processing on the neighbor model update vectors within each directional similar update group based on the effective independent support weights, to obtain the intra-group neighbor consensus update corresponding to each directional similar update group; then, based on the effective independent support weights of each directional similar update group, it performs inter-group weighted processing on the intra-group neighbor consensus update to obtain the neighbor-side consensus update.

[0123] The processing terminal calculates the directional consistency between the local model update data and the neighbor consensus update, and generates a local reference weight and a neighbor receiving weight based on the directional consistency. For example, a preset consistency threshold of 0.60 is used. When the directional consistency reaches 0.60, the processing terminal increases the neighbor receiving weight; when the directional consistency does not reach 0.60, the processing terminal increases the local reference weight. The processing terminal performs a weighted synthesis of the local model update data and the neighbor consensus update based on the local reference weight and the neighbor receiving weight to obtain a local consensus update. Subsequently, the processing terminal updates the current local model based on the local consensus update to obtain the local model update result.

[0124] After the above processing, the processing terminal outputs the local consensus update and local model update results, which serve as the model update results for the current training round and as the data basis for generating subsequent outgoing model update messages.

[0125] Example: For instance, the effective independent support weight of the first direction similarity update group is 1.22, and the effective independent support weight of the second direction similarity update group is 1.00. The processing terminal generates the first group intra-neighbor consensus update based on the neighbor model update vectors of neighbor nodes A and B in the first direction similarity update group, and generates the second group intra-neighbor consensus update based on the neighbor model update vector of neighbor node C in the second direction similarity update group.

[0126] The processing terminal then performs inter-group weighted processing on the neighbor consensus updates within the first and second groups based on the effective independent support weights of the two directionally similar update groups, obtaining the neighbor-side consensus update. If the directional consistency between the local model update data and the neighbor-side consensus update reaches a preset consistency threshold, the processing terminal increases the neighbor-side receiving weight; if it does not reach the preset consistency threshold, the processing terminal increases the local reference weight. Assuming the local reference weight is 0.40 and the neighbor-side receiving weight is 0.60, the processing terminal performs weighted synthesis on the local model update data and the neighbor-side consensus update according to these weights to obtain a local consensus update, and updates the current local model based on the local consensus update, obtaining the local model update result for the 35th training round.

[0127] Through the above technical solution, this application solves the problem of lack of reference constraints between neighbor-side consensus update and local model update data. By first generating neighbor-side consensus update based on effective independent support weights, then calculating the directional consistency between local model update data and neighbor-side consensus update, and generating local reference weights and neighbor-side receiving weights based on directional consistency, the weighted synthesis of local model update data and neighbor-side consensus update is realized, thereby obtaining a local consensus update for updating the current local model.

[0128] This application further proposes that after updating the current local model based on local consensus, it also includes:

[0129] Obtain the directional similarity update groups participating in robust aggregation in the current training round, the multi-hop echo amplification index of each directional similarity update group, and the propagation source summary of neighbor model update messages within each directional similarity update group;

[0130] When the multi-hop echo amplification index of the directional similarity update group reaches the preset echo compression threshold, the summary items representing the upstream update source in each propagation source summary within the directional similarity update group are obtained, and the summary items are merged to obtain the equivalent propagation source summary covering the upstream update source within the directional similarity update group.

[0131] In one embodiment, a digest term refers to the smallest labeled unit in the propagation source digest used to characterize the upstream update source. The propagation source digest can be represented using a bitmap digest, a hash set digest, or a Bloom filter digest. When the propagation source digest uses a bitmap digest, the digest term is a label bit in the bitmap, and a label bit with a valid label value indicates that the corresponding upstream update source has been absorbed by the neighbor model update message; when the propagation source digest uses a hash set digest, the digest term is a hash value in the hash set, and each hash value corresponds to a compressed identifier of an upstream update source; when the propagation source digest uses a Bloom filter digest, the digest term is a label bit in the Bloom filter that is mapped to a valid state.

[0132] Based on the difference between the local model update result and the current local model, the difference in parameter vectors is used as the outgoing model update vector;

[0133] The equivalent propagation source summary is used as the outward propagation source summary, and the outward propagation hop count summary is generated based on the propagation hop count summary of the current training round. The sending trigger time of the outward model update message is recorded, and the sending time information is generated.

[0134] The outgoing model update vector, outgoing propagation source summary, outgoing propagation hop count summary, and sending time information are assembled into the outgoing model update message for the next training round, and the outgoing model update message is output to neighboring nodes.

[0135] Specifically, after updating the current local model based on local consensus, the processing terminal obtains the local model update result, the current local model, the directional similarity update groups participating in robust aggregation in the current training round, the multi-hop echo amplification index of each directional similarity update group, and the propagation source summary of neighbor model update messages within each directional similarity update group.

[0136] The processing terminal determines whether the multi-hop echo amplification index of the directional similarity update group reaches the preset echo compression threshold. If it does, it obtains the summary items representing the upstream update sources in each propagation source summary within the directional similarity update group and performs merging processing on the summary items to obtain an equivalent propagation source summary covering the upstream update sources within the directional similarity update group. If it does not reach the threshold, it uses the propagation source summary within the directional similarity update group as an uncompressed propagation source summary to participate in the generation of subsequent outgoing propagation source summaries.

[0137] For example, the preset echo compression threshold can be 1.50; when the multi-hop echo amplification index of the directional similarity update group reaches 1.50, the processing terminal performs summary item merging processing on the propagation source summary in the directional similarity update group to obtain an equivalent propagation source summary; when it does not reach 1.50, the summary item merging processing is not performed.

[0138] The processing terminal generates an outgoing model update vector based on the parameter difference between the local model update result and the current local model; and uses the equivalent propagation source summary as the outgoing propagation source summary. The processing terminal further obtains the propagation hop count summary corresponding to the neighbor model update messages participating in robust aggregation in the current training round, and generates an outgoing propagation hop count summary based on the propagation hop count represented by the propagation hop count summary.

[0139] The processing terminal assembles the outgoing model update vector, the outgoing propagation source summary, the outgoing propagation hop count summary, and the sending time information into the outgoing model update message for the next training round, and outputs the outgoing model update message to neighboring nodes.

[0140] After the above processing, the processing terminal outputs the outgoing model update message for the next training round, so that the propagation sources that have completed source merging in the current training round can continue to participate in the propagation source overlap calculation as outgoing propagation source summaries in the next training round.

[0141] Example: For instance, the processing terminal obtains the local model update result in the 35th training round and acquires the first direction similarity update group and the second direction similarity update group. The multi-hop echo amplification index of the first direction similarity update group is 1.64, the multi-hop echo amplification index of the second direction similarity update group is 1.00, and the preset echo compression threshold is 1.50.

[0142] The processing terminal determines that the multi-hop echo amplification index of the first direction similar update group reaches the preset echo compression threshold. Therefore, it obtains the propagation source summaries corresponding to neighbor node A and neighbor node B within the first direction similar update group, and performs a merging process on the summary items representing the upstream update source in the two propagation source summaries to obtain the equivalent propagation source summary of the first direction similar update group. The processing terminal determines that the multi-hop echo amplification index of the second direction similar update group does not reach the preset echo compression threshold, therefore, it does not perform source merging processing on the second direction similar update group.

[0143] The processing terminal generates a 120,000-dimensional outgoing model update vector based on the parameter difference between the local model update result of the 35th training round and the current local model; it uses the equivalent propagation source summary of the first direction similar update group as the outgoing propagation source summary; and it generates an outgoing propagation hop count summary based on the propagation hop count summary corresponding to the neighbor model update messages participating in robust aggregation in the current training round.

[0144] Finally, the processing terminal assembles the 120,000-dimensional outgoing model update vector, outgoing propagation source summary, outgoing propagation hop count summary, and sending time information into the outgoing model update message for the 36th training round, and outputs it to neighboring nodes.

[0145] Through the above technical solution, this application solves the problem that after the current processing terminal completes echo decorrelation compression, the outgoing model update message may still carry unmerged propagation sources, resulting in duplicate weighting downstream. By generating an equivalent propagation source summary based on the propagation source summaries in the direction similarity update group that have reached the preset echo compression threshold after the local model update result is generated, and writing the equivalent propagation source summary as the outgoing propagation source summary into the outgoing model update message, the propagation sources that have completed source merging can continue to participate in the propagation source overlap calculation in the next training round.

[0146] This application further proposes that after assembling the outward model update vector, outward propagation source summary, outward propagation hop count summary, and sending time information into the outward model update message for the next training round, it also includes:

[0147] Obtain the outward propagation source summary from the outward model update message, and obtain the multi-hop echo amplification index and effective independent support weight of the directional similarity update group corresponding to the generated outward propagation source summary;

[0148] The source compression status flag is generated based on the multi-hop echo amplification index to be used to write the propagation link metadata of the outgoing model update message; when the multi-hop echo amplification index reaches the preset echo flag threshold, the source compression status flag is determined as an echo compression flag; when the multi-hop echo amplification index does not reach the preset echo flag threshold, the source compression status flag is determined as a normal propagation flag.

[0149] Obtain the sum of effective independent support weights for each directional similarity update group within the current training round, and determine the ratio of the effective independent support weights of the directional similarity update group corresponding to the generated outward propagation source summary to the sum of effective independent support weights as the outward source reception weight;

[0150] Write the source compression status flag and the outgoing source receiving weight into the propagation link metadata of the outgoing model update message, and output the written outgoing model update message to the neighboring nodes.

[0151] Specifically, after generating the outgoing model update message for the next training round, the processing terminal obtains the outgoing propagation source summary in the outgoing model update message, and obtains the multi-hop echo amplification index and effective independent support weight of the direction similarity update group corresponding to the generation of the outgoing propagation source summary.

[0152] The processing terminal determines whether the multi-hop echo amplification index of the directional similarity update group reaches a preset echo label threshold. If it does, the source compression state label is determined as an echo compression label; if it does not, the source compression state label is determined as a normal propagation label. The source compression state label is used to indicate whether the outward propagation source summary is generated by a directional similarity update group that meets the echo compression condition.

[0153] For example, the preset echo tag threshold is 1.50; when the multi-hop echo amplification index of the direction similarity update group corresponding to the generated outward propagation source summary reaches 1.50, the processing terminal determines the source compression status tag as an echo compression tag; when it does not reach 1.50, the processing terminal determines the source compression status tag as a normal propagation tag.

[0154] The processing terminal obtains the sum of the effective independent support weights of each directional similarity update group within the current training round, and calculates the ratio between the effective independent support weight of the directional similarity update group corresponding to the generated outward propagation source summary and the sum of the effective independent support weights to obtain the outward source reception weight. The outward source reception weight is used to represent the proportion of the outward propagation source summary in the effective independent support of the current training round.

[0155] The processing terminal writes the source compression status flag and the outgoing source receiving weight into the propagation link metadata of the outgoing model update message, and outputs the written outgoing model update message to the neighboring nodes.

[0156] After the above processing, the processing terminal outputs an outward model update message containing an outward model update vector, an outward propagation source summary, an outward propagation hop count summary, transmission time information, a source compression status marker, and outward source reception weights. This message serves as the data basis for neighboring nodes to receive and calculate propagation source overlap, arrival time chain consistency, and basic support weights in the next training round.

[0157] Example: For instance, the processing terminal has generated an outgoing model update message for the 36th training round in the 35th training round. This outgoing model update message includes a 120,000-dimensional outgoing model update vector, a 256-bit outgoing propagation source summary, an outgoing propagation hop count summary, and sending time information.

[0158] The outward propagation source summary is generated from the equivalent propagation source summary of the first direction similarity update group. The multi-hop echo amplification index of the first direction similarity update group is 1.64, and the effective independent support weight is 1.22; the effective independent support weight of the second direction similarity update group is 1.00; and the preset echo tagging threshold is 1.50.

[0159] The processing terminal determines that the multi-hop echo amplification index of 1.64 for the first direction similarity update group reaches the preset echo labeling threshold of 1.50, and therefore identifies the source compression state as an echo compression label. The processing terminal calculates the ratio of the effective independent support weight of 1.22 for the first direction similarity update group to the sum of the effective independent support weights of all direction similarity update groups in the current training round, which is 2.22, and obtains an outward source reception weight of approximately 0.55.

[0160] Finally, the processing terminal writes the echo compression tag and the outgoing source receiving weight of 0.55 into the propagation link metadata of the outgoing model update message, and outputs the written outgoing model update message to the neighboring nodes.

[0161] Through the above technical solution, this application solves the problem that outgoing model update messages only carry a propagation source digest but cannot characterize whether the source digest has undergone echo compression processing. It generates a source compression status marker based on the multi-hop echo amplification index of the similar update group corresponding to the generated outgoing propagation source digest, and generates an outgoing source reception weight based on the effective independent support weight of the similar update group in that direction. Then, it writes the two into the propagation link metadata of the outgoing model update message, so that neighboring nodes can identify the compression status and reception weight of the outgoing propagation source digest after receiving the message.

[0162] This application further proposes, after processing the neighbor model update message containing the source compressed status flag and the received weight of the outgoing source, that the terminal receives the message, it also includes:

[0163] Extract the neighbor model update vector, outward propagation source summary, outward propagation hop count summary, arrival time information, source compression status flag, and outward source reception weight from the neighbor model update message;

[0164] The outward propagation source summary is used as the propagation source summary, the outward propagation hop count summary is used as the propagation hop count summary, and the propagation source overlap is generated based on the propagation source summary. The arrival time sequence consistency is generated based on the propagation hop count summary and arrival time information.

[0165] Obtain the basic support weights corresponding to the neighbor model update messages;

[0166] When the source compression status is marked as echo compression, the base support weight is multiplied by the outgoing source receiving weight to obtain the corrected base support weight.

[0167] When the source compression state is marked as a normal propagation mark, the base support weight is used as the corrected base support weight;

[0168] The original support weights of the directional similarity update group are generated based on the corrected basic support weights corresponding to the model update messages of each neighbor model within the same directional similarity update group. The original support weights are then compressed based on the multi-hop echo amplification index of the directional similarity update group to obtain effective independent support weights.

[0169] Specifically, after receiving a neighbor model update message in the current training round, the processing terminal extracts the neighbor model update vector, outward propagation source summary, outward propagation hop count summary, arrival time information, source compression status flag, and outward source reception weight from the neighbor model update message. The outward propagation source summary is used as the propagation source summary in this round of calculation, the outward propagation hop count summary is used as the propagation hop count summary in this round of calculation, the source compression status flag indicates whether the propagation source in the neighbor model update message has undergone source compression processing, and the outward source reception weight is used to correct the basic support weights corresponding to the neighbor model update message.

[0170] The processing terminal uses the outgoing propagation source digest as the propagation source digest and the outgoing propagation hop count digest as the propagation hop count digest. It also uses the propagation source digest to generate the propagation source overlap and the propagation hop count digest and arrival time information to generate the arrival time sequence consistency.

[0171] The processing terminal obtains the basic support weights corresponding to the neighbor model update messages and determines the type of the source compression status flag. When the source compression status flag is an echo compression flag, the processing terminal multiplies the basic support weights by the outgoing source receiving weights to obtain the corrected basic support weights; when the source compression status flag is a normal propagation flag, the processing terminal uses the basic support weights as the corrected basic support weights.

[0172] The processing terminal sums the modified basic support weights corresponding to the model update messages of each neighbor in the same direction similarity update group to obtain the original support weights of the direction similarity update group; then, it performs compression processing on the original support weights based on the multi-hop echo amplification index of the direction similarity update group to obtain the effective independent support weights.

[0173] After the above processing, the processing terminal outputs a propagation source summary, a propagation hop count summary, a corrected basic support weight, the original support weight of the direction similarity update group, and an effective independent support weight, which serve as the data basis for subsequent generation of neighbor-side consensus updates and local consensus updates.

[0174] Example: For instance, the processing terminal receives a neighbor model update message from neighbor node E in the 36th training round. This neighbor model update message includes a 120,000-dimensional neighbor model update vector, a 256-bit outward propagation source summary, an outward propagation hop count summary, arrival time information, a source compression status flag, and an outward source reception weight of 0.55.

[0175] The processing terminal uses the outgoing propagation source digest as the propagation source digest and the outgoing propagation hop count digest as the propagation hop count digest, and uses both to generate propagation source overlap and arrival time chain consistency in subsequent calculations. If the basic support weight of the neighbor model update message is 1.00 and the source compression status is marked as echo compression, the processing terminal multiplies the basic support weight 1.00 by the outgoing source reception weight 0.55 to obtain the corrected basic support weight 0.55.

[0176] If the neighbor model update messages corresponding to neighbor node E and neighbor node F are assigned to the same directional similarity update group, and the corrected basic support weight corresponding to neighbor node F is 0.80, then the processing terminal sums 0.55 and 0.80 to obtain the original support weight of 1.35 for the directional similarity update group. If the multi-hop echo amplification index of the directional similarity update group is 1.50, then the processing terminal performs compression processing on the original support weight 1.35 based on the multi-hop echo amplification index to obtain an effective independent support weight of 0.90.

[0177] Through the above technical solution, this application solves the problem that downstream processing terminals still calculate basic support weights as ordinary messages after receiving neighbor model update messages that have completed source compression. By extracting the source compression status flag and the outgoing source receiving weight from the neighbor model update message, and using the outgoing source receiving weight to correct the basic support weight when the source compression status flag is the echo compression flag, the echo compression result of the previous training round can continue to be used in the calculation of the original support weight and effective independent support weight of the directional similarity update group in this round, thereby reducing the risk that the same propagating echo will be amplified again in the downstream node.

[0178] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A decentralized federated learning Byzantine robust consensus method, characterized in that, Robust training of machine learning models for decentralized federated learning, performed by a processing terminal, includes: Get the current local model, local model update data, and neighbor model update message set composed of neighbor model update messages for the current training round. The current local model is the local machine learning model, and the neighbor model update message includes the neighbor model update vector, propagation source summary, propagation hop count summary, and arrival time information. Based on the neighbor model update vector, an update direction signature representing its directional attribute is generated, and based on the directional similarity between update direction signatures, neighbor model update messages that meet the preset directional similarity conditions are divided into directional similarity update groups. Based on the direction similarity, the update direction signature of each neighbor model update message in the group is updated to generate the direction similarity within the group. Based on the corresponding propagation source summary, the propagation source overlap is generated. Based on the corresponding propagation hop count summary and arrival time information, the arrival time sequence consistency is generated. Based on intra-group directional similarity, propagation source overlap, arrival time chain consistency, and the number of messages within the directional similarity update group, a multi-hop echo amplification index is generated to represent the degree of non-independent support of neighbor model update messages within the directional similarity update group. Based on the multi-hop echo amplification index, decorrelation compression is performed on the original support weights of the directional similarity update group to obtain effective independent support weights; Robust aggregation of neighbor model update vectors is performed based on effective independent support weights, and robust aggregation is also performed in conjunction with local model update data to obtain local consensus updates. The current local model is then updated based on the local consensus updates.

2. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of obtaining the directional similarity update group includes: Obtain the update direction signature of each neighbor model update message in the neighbor model update message set; The inner product between the update direction signatures corresponding to the update messages of any two neighbor models is calculated by vector inner product operation; When the inner product result is not less than zero, the inner product result is determined as the directional similarity between the update messages of the corresponding two neighbor models; When the inner product result is less than zero, the directional similarity between the update messages of the corresponding two neighbor models is set to zero. Establish a directional association between update messages of two neighboring models whose directional similarity reaches a preset directional similarity threshold; Based on directional association, connected subsets are determined in the neighbor model update message set, and these connected subsets are used as directional similarity update groups. Among them, neighbor model update messages in the same direction similar update group are directly or indirectly related through directional association.

3. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of generating the overlap of propagation sources includes: Obtain a summary of the propagation sources of update messages from neighboring models within the direction similarity update group; Calculate the size of the intersection and union of the summaries of any two propagation source summaries; Based on the ratio between the size of the intersection of the summaries and the size of the union of the summaries, the source overlap between the update messages of the corresponding two neighbor models is obtained; The average overlap of each source within the directional similarity update group is processed to obtain the propagation source overlap of the directional similarity update group. Among them, the propagation source digest is the digest data obtained by compressing and encoding the upstream update source absorbed by the neighbor model update message.

4. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of generating a consistent arrival time chain includes: Obtain the propagation hop count summary and arrival time information of each neighbor model update message in the direction similarity update group; Based on the historical communication records of the processing terminal, generate the normal propagation delay and propagation delay fluctuation scale corresponding to the number of propagation hops per unit; Arrival time difference is generated based on the absolute value of the difference between the arrival time information of the update messages of any two neighbor models, and propagation hop difference is generated based on the absolute value of the difference between the propagation hop count represented by the propagation hop count digest of the update messages of any two neighbor models. Based on the arrival time difference, propagation hop difference, regular propagation delay, and propagation delay fluctuation scale, the temporal chain consistency value between the update messages of the corresponding two neighbor models is obtained; Timing chain consistency value satisfy: ; in, , This indicates the arrival time information of the neighbor model update messages i and j. , This represents the corresponding propagation hop count summary. Indicates the normal propagation delay. Indicates the scale of propagation delay fluctuation. This represents a pre-defined minimum positive number; The consistency values ​​of each time-series chain within the direction-similar update group are averaged to obtain the consistency of the arrival time-series chain of the direction-similar update group.

5. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of generating the multi-hop echo amplification index includes: Get the intra-group directional similarity, propagation source overlap, arrival time chain consistency, and message count of the directional similarity update group; The echo correlation strength is obtained by performing multiplicative coupling processing on intragroup directional similarity, propagation source overlap, and arrival time sequence consistency. A multi-hop echo amplification index is generated based on echo correlation strength and message quantity; Multi-jump echo amplification index satisfy: ; in, Indicates the number of messages. Indicates intra-group directional similarity. Indicates the degree of overlap in the sources of transmission. This indicates consistency of the arrival time chain; Among them, echo correlation strength is used to characterize the degree to which neighbor model update messages in the same direction similarity update group meet the common requirements in three dimensions: direction consistency, source overlap, and temporal chain matching.

6. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of obtaining effective independent support weights includes: Obtain the basic support weights for the model update messages of each neighbor model within the direction similarity update group; Sum the basic support weights within the directional similarity update group to obtain the original support weights of the directional similarity update group. The multi-hop echo amplification index is used as a non-independent compression factor for the directional similarity update group to compress the original support weights, resulting in effective independent support weights. Effective independent support weights satisfy: ; in, Indicates a direction similarity update group. Indicates the basic support weight. This indicates the multi-hop echo amplification index.

7. The decentralized federated learning Byzantine robust consensus method according to claim 1, characterized in that, The process of obtaining a local consensus update includes: Obtain the effective independent support weights of each directional similarity update group, the neighbor model update vectors within each directional similarity update group, and the local model update data for the current training round; Based on the effective independent support weights, the neighbor model update vectors in each directional similarity update group are weighted within the group to obtain the intra-group neighbor consensus update corresponding to each directional similarity update group. Based on the effective independent support weights of similar update groups in each direction, inter-group weighted processing is performed on the neighbor consensus update within each group to obtain the neighbor-side consensus update. Calculate the directional consistency between the local model update data and the neighbor consensus update, generate local reference weights and neighbor receiving weights based on the directional consistency, and perform weighted synthesis on the local model update data and the neighbor consensus update according to the local reference weights and the neighbor receiving weights to obtain the local consensus update; The local model is updated based on local consensus updates to obtain the local model update result.

8. The decentralized federated learning Byzantine robust consensus method according to claim 7, characterized in that, After updating the current local model based on local consensus, it also includes: Obtain the directional similarity update groups participating in robust aggregation in the current training round, the multi-hop echo amplification index of each directional similarity update group, and the propagation source summary of neighbor model update messages within each directional similarity update group; When the multi-hop echo amplification index of the directional similarity update group reaches the preset echo compression threshold, the summary items representing the upstream update source in each propagation source summary within the directional similarity update group are obtained, and the summary items are merged to obtain the equivalent propagation source summary covering the upstream update source within the directional similarity update group. Generate an outgoing model update vector based on the difference between the local model update result and the current local model. The equivalent propagation source summary is used as the outward propagation source summary, and the outward propagation hop count summary is generated based on the propagation hop count summary of the current training round. The sending trigger time of the outward model update message is recorded, and the sending time information is generated. The outgoing model update vector, outgoing propagation source summary, outgoing propagation hop count summary, and sending time information are assembled into the outgoing model update message for the next training round, and the outgoing model update message is output to neighboring nodes.

9. The decentralized federated learning Byzantine robust consensus method according to claim 8, characterized in that, After assembling the outward model update vector, outward propagation source summary, outward propagation hop count summary, and sending time information into the outward model update message for the next training round, it also includes: Obtain the outward propagation source summary from the outward model update message, and obtain the multi-hop echo amplification index and effective independent support weight of the directional similarity update group corresponding to the generated outward propagation source summary; The source compression status flag is generated based on the multi-hop echo amplification index to be used to write the propagation link metadata of the outgoing model update message; when the multi-hop echo amplification index reaches the preset echo flag threshold, the source compression status flag is determined as an echo compression flag; when the multi-hop echo amplification index does not reach the preset echo flag threshold, the source compression status flag is determined as a normal propagation flag. Obtain the sum of effective independent support weights for each directional similarity update group within the current training round, and determine the ratio of the effective independent support weights of the directional similarity update group corresponding to the generated outward propagation source summary to the sum of effective independent support weights as the outward source reception weight; Write the source compression status flag and the outgoing source receiving weight into the propagation link metadata of the outgoing model update message, and output the written outgoing model update message to the neighboring nodes.

10. The decentralized federated learning Byzantine robust consensus method according to claim 9, characterized in that, After processing the neighbor model update message that the terminal receives and writes the source compression status flag and the outgoing source received weight, it also includes: Extract the neighbor model update vector, outward propagation source summary, outward propagation hop count summary, arrival time information, source compression status flag, and outward source reception weight from the neighbor model update message; The outgoing propagation source summary is used as the propagation source summary, the outgoing propagation hop count summary is used as the propagation hop count summary, and the propagation source overlap is generated based on the propagation source summary. The arrival time sequence consistency is generated based on the propagation hop count summary and arrival time information. Obtain the basic support weights corresponding to the neighbor model update messages; When the source compression status is marked as echo compression, the base support weight is multiplied by the outgoing source receiving weight to obtain the corrected base support weight. When the source compression state is marked as a normal propagation mark, the base support weight is used as the corrected base support weight; The original support weights of the directional similarity update group are generated based on the corrected basic support weights corresponding to the model update messages of each neighbor model within the same directional similarity update group. The original support weights are then compressed based on the multi-hop echo amplification index of the directional similarity update group to obtain effective independent support weights.

Images