A neighbor projection gradient coordination compression method and device and a federated learning system

Abstract

The application provides a kind of near neighbor projection type gradient coordination compression method, device and federal learning system, it is related to federal learning technical field.The application obtains the gradient uploaded by client and carries out direction normalization, carries out the direction embedding update of client by exponential moving average, then the similarity between client is calculated and near neighbor set is generated, constructs similar graph, then executes conflict detection and weight setting, executes weighted orthogonal projection in the near neighbor direction of each client freezing to correct gradient, finally, weighted aggregation gradient and update global model parameter.The application constructs client similar graph and combines near neighbor projection mechanism, efficiently detects and corrects gradient conflict in local range, avoids the problem of high complexity and excessive information reduction caused by global processing, improves global model convergence speed, stability and precision without increasing additional calculation and communication overhead of client, and is suitable for large-scale non-independent and identically distributed data scene.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more specifically, to a nearest-neighbor projection gradient coordination compression method, apparatus, and federated learning system. Background Technology

[0002] In distributed training scenarios such as federated learning, gradient conflicts are one of the core issues leading to performance degradation, especially under non-independent and identically distributed (Non-IID) data conditions. When there are significant conflicts in the gradient directions uploaded by the client, the aggregated global gradients may cancel each other out, resulting in inefficient model updates, slower convergence, and unstable training.

[0003] Existing technical solutions mainly address gradient conflict from two directions: multi-task learning and federated learning. In multi-task learning, gradient projection methods such as PCGrad (Projected Conflicting Gradients) and RI-PCGrad (Rescaled and Inverted PCGrad) reduce mutual interference by detecting cases where the gradient cosine similarity is negative and performing orthogonal projection or rescaling operations. Methods such as CAGrad (Conflict-Aware Gradient) achieve soft avoidance of conflict by optimizing the balance between target conflict and task performance. In the context of federated learning, methods such as FedFV (Federated Gradient Feature Vector) attempt to detect conflicts using cosine similarity and reduce gradient cancellation through direction or magnitude correction; Gradient Calibration proposes a calibration mechanism for Non-IID data, scaling or reconstructing gradients uploaded by the client to reduce the impact of conflicts; FedLAG (Federated Layer-wise Gradient Aggregation) is based on hierarchical conflict analysis, removing layers with severe conflicts from global aggregation to achieve personalized training.

[0004] While these methods can mitigate the performance loss caused by gradient conflicts to some extent, they still have significant shortcomings. First, most methods employ global or heuristic conflict detection and correction strategies, lacking modeling of similarity relationships between clients, resulting in a lack of targeted conflict handling. Second, existing methods typically reduce or eliminate conflict components in a coarse-grained manner, easily leading to excessive loss of useful gradient information, thus affecting model convergence quality. Third, when the number of clients is large or the data distribution is highly heterogeneous, existing methods suffer from high computational complexity and insufficient stability, making them difficult to apply efficiently in large-scale federated learning environments. Furthermore, these methods are mostly single-step or static corrections, lacking dynamic adaptive mechanisms, making it difficult to address the optimization needs of different training stages.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention aims to provide a nearest-neighbor projection gradient coordination compression method, device, and federated learning system to address the shortcomings of existing gradient conflict handling methods in distributed training scenarios such as federated learning, and to solve the gradient direction conflict problem caused by differences in data distribution.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] A nearest-neighbor projection gradient-coordinated compression method includes:

[0009] S1, obtain the gradient vector uploaded by each client and normalize its direction;

[0010] S2, based on the normalized gradient vector, updates the orientation embedding of each client through exponential moving average;

[0011] S3, calculate the similarity between clients based on the directional embedding and select several nearest neighbor clients with the highest similarity to generate a nearest neighbor set;

[0012] S4, perform conflict detection based on similarity and the nearest neighbor set to obtain a set of conflict edges;

[0013] S5, based on the set of conflicting edges, perform weighted orthogonal projection on the nearest neighbor direction frozen by each client to correct the gradient;

[0014] S6 updates the global model parameters after weighted aggregation of the corrected gradients.

[0015] Preferably, the formula for normalizing the gradient vector by direction is:

[0016] ;

[0017] in, In the t-th round of federated learning, the client... The normalized gradient vector; In the t-th round of federated learning, the client... The gradient vector; To represent extremely small numbers and prevent division by zero; express The Euclidean norm is used to measure the length of the gradient vector.

[0018] Preferably, the expression for updating the orientation embedding of each client using exponential moving average is:

[0019] ;

[0020] And embed the orientation into a unit sphere:

[0021] ;

[0022] in, In the t-th round of federated learning, the client... Directional embedding; Indicating the client in the previous round of federated learning Directional embedding; In the t-th round of federated learning, the client... The normalized gradient vector; For smoothing coefficients; It represents a very small number.

[0023] Preferably, the similarity between clients is obtained by calculating the cosine similarity of the directional embeddings of each pair of clients, expressed as:

[0024] ;

[0025] in, This represents the directional cosine similarity between clients i and j in the t-th round of federated learning; , In the t-th round of federated learning, the client... The direction of embedding; j is the neighbor of client i.

[0026] Preferably, the collision detection specifically includes:

[0027] Based on the similarity between clients and the nearest neighbor set, if the similarity is less than a negative threshold, it is determined to be a conflict edge, and the formula is as follows:

[0028] ;

[0029] in, In the t-th round of federated learning, the client... The conflict edge; Indicates client The nearest neighbor set; In the t-th round of federated learning, the client... , Directional cosine similarity; The similarity threshold;

[0030] Next, weights are assigned to the conflicting edges:

[0031] ;

[0032] in, For the client , The weights of the conflicting edges; This is the upper limit of the weight, used to prevent a single neighbor from becoming overly dominant.

[0033] Preferably, the expression for the weighted orthogonal projection is:

[0034] ;

[0035] in, In the t-th round of federated learning, the client... The gradient vector; Indicates the weighted orthogonal projection , that is, the corrected gradient vector; The projection intensity coefficient; In the t-th round of federated learning, the client... The conflict edge; For the client , The weights of the conflicting edges; This represents the normalized gradient vector after freezing in the nearest neighbor direction; In the t-th round of federated learning, the client... The normalized gradient vector; Represents a very small number; This represents the direction cosine similarity between the two.

[0036] Preferably, the corrected gradient is weighted and averaged, as expressed by:

[0037] ;

[0038] in, The global gradient after weighted aggregation in the t-th round of federated learning; The weight of the client; K represents the total number of clients; This represents the corrected gradient vector.

[0039] Preferably, the expression for updating the global model parameters is:

[0040] ;

[0041] in, , These are the global model parameters for the (t+1)th and tth rounds of federated learning, respectively; This is the learning rate.

[0042] The present invention also provides a nearest-neighbor projection gradient coordinated compression device, comprising:

[0043] The orientation normalization module is used to obtain the gradient vector uploaded by each client and perform orientation normalization.

[0044] The exponential sliding update module is used to update the orientation embedding of each client based on the normalized gradient vector by exponential sliding average.

[0045] The nearest neighbor set generation module is used to calculate the similarity between clients based on the directional embedding and select the most similar nearest neighbor clients to generate a nearest neighbor set;

[0046] The conflict detection module is used to perform conflict detection based on similarity and the nearest neighbor set to obtain a set of conflicting edges;

[0047] The gradient correction module is used to perform a weighted orthogonal projection on the nearest neighbor directions frozen by each client to correct the gradient based on the set of conflicting edges;

[0048] The aggregation update module is used to update the global model parameters after weighted aggregation of the corrected gradients.

[0049] The present invention also provides a federated learning system, including multiple clients and a server. Each client completes local model training and updates and uploads gradient vectors to the server. The server includes a processor and a memory. The memory stores a computer program that can be executed by the processor to implement a nearest-neighbor projection gradient coordination compressor and its compression method as described above.

[0050] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device on which the computer-readable storage medium resides, implement a nearest-neighbor projection gradient coordinated compression method as described above.

[0051] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0052] This invention utilizes a client-side similarity graph to limit conflict handling to the nearest neighbor set, avoiding the high computational complexity associated with global processing. By updating the direction embedding through exponential moving average, the stability of the client-side direction representation is enhanced, thereby improving the accuracy of conflict detection. This invention employs a nearest neighbor projection mechanism to remove conflict components while preserving useful gradient information to the maximum extent, avoiding the performance loss caused by excessive shaving in traditional methods and addressing the shortcomings of existing technologies in gradient conflict handling. Furthermore, by dynamically adjusting the projection intensity coefficient and weights, it can adaptively adapt to the needs of different training stages, further improving the optimization effect.

[0053] This invention demonstrates significant advantages in large-scale federated learning environments, achieving efficient gradient conflict mitigation without increasing additional computational and communication overhead on the client side. Through selective projection of local nearest neighbors, it significantly reduces computational complexity and exhibits good scalability. Under conditions of non-independent and identically distributed data, the method of this invention can significantly accelerate model convergence, improve the accuracy and robustness of the global model, and enhance the convergence efficiency and stability of federated learning systems, providing important technical support for the field of distributed machine learning. Attached Figure Description

[0054] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is a schematic diagram of a nearest-neighbor projection gradient coordinated compression method provided in Example 1.

[0056] Figure 2 This is a schematic diagram of a proximity projection gradient coordinated compression device provided in Embodiment 2.

[0057] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0059] Example 1

[0060] Embodiment 1 of the present invention provides a nearest-neighbor projection gradient coordinated compressor and its compression method, which can be implemented by a nearest-neighbor projection gradient coordinated compression device (hereinafter referred to as compression device), and in particular, executed by one or more processors within the compression device.

[0061] In this embodiment, the compression device may be an electronic device equipped with a processor, the processor having a computer program for the nearest-neighbor projection gradient coordinated compression method and the computer program being executable, such as a computer, smartphone, smart tablet, workstation, etc., without limitation.

[0062] In federated learning systems, multiple clients participate in distributed training. Each client calculates gradients based on its local data and uploads them to the server for aggregation. However, due to the significant differences in the distribution of client data, the uploaded gradient directions may conflict significantly, leading to inefficient global model updates and even performance degradation. To address this issue, such as... Figure 1 As shown, a nearest-neighbor projection gradient coordination compression method is applied to the server side of a federated learning system, which includes steps S1 to S6.

[0063] S1: Obtain the gradient vector uploaded by each client and normalize its direction.

[0064] In each round of federated learning, the client first completes local training and computes the gradient vector, then uploads the gradient to the server. Different clients may have significantly different gradient magnitudes due to varying training data volumes, but the gradient direction is more valuable for indicating the model's convergence direction. To eliminate the impact of gradient scale differences between clients on subsequent processing, the server performs direction normalization on the uploaded gradients.

[0065] The formula for normalizing the gradient vector by direction is:

[0066] ;

[0067] in, In the t-th round of federated learning, the client... The normalized gradient vector; In the t-th round of federated learning, the client... The gradient vector; To represent extremely small numbers and prevent division by zero; express The Euclidean norm is used to measure the length of the gradient vector; This represents the Euclidean norm.

[0068] S2, based on normalized gradient vectors, updates the orientation embedding of each client through exponential moving average.

[0069] Exponential moving average is a commonly used method for smoothing data. Updating the directional embedding in this way better captures the long-term trend of the client's gradient direction and reduces the impact of short-term fluctuations. In federated learning, client data may change dynamically, and this smoothing process helps stabilize the model's training process.

[0070] The expression for updating the orientation embedding for each client using the exponential moving average is:

[0071] ;

[0072] And embed the orientation into a unit sphere:

[0073] ;

[0074] in, In the t-th round of federated learning, the client... Directional embedding; Indicating the client in the previous round of federated learning Directional embedding; In the t-th round of federated learning, the client... The normalized gradient vector; For smoothing coefficients; It represents a very small number.

[0075] S3, calculate the similarity between clients based on the directional embedding and select the nearest neighbor clients with the highest similarity to generate a nearest neighbor set.

[0076] The similarity between clients can be obtained by calculating the cosine similarity of the directional embeddings of each pair of clients, as expressed by:

[0077] ;

[0078] in, This represents the directional cosine similarity between clients i and j in the t-th round of federated learning; , In the t-th round of federated learning, the client... Directional embedding; j is the client The neighbors.

[0079] For each client Select the k nearest neighbor clients with the highest similarity to form a nearest neighbor set. .

[0080] Calculating the similarity between clients and selecting nearest neighbors aims to find a set of clients that are similar in the gradient direction. In federated learning, clients with similar gradient directions often imply similarities in their data distribution, features, and other aspects. By focusing on these nearest neighbors, gradient coordination and information aggregation can be performed more effectively.

[0081] S4. Based on similarity and the nearest neighbor set, perform conflict detection to obtain a set of conflict edges.

[0082] Specifically, the conflict detection includes:

[0083] Based on the similarity between clients and the nearest neighbor set, if the similarity is less than a negative threshold, it is determined to be a conflict edge, and the formula is as follows:

[0084] ;

[0085] in, In the t-th round of federated learning, the client... The conflict edge; Indicates client The nearest neighbor set; In the t-th round of federated learning, the client... , Directional cosine similarity; The similarity threshold;

[0086] Next, weights are assigned to the conflicting edges:

[0087] ;

[0088] in, For the client , The weights of the conflicting edges; This is the upper limit of the weight, used to prevent a single neighbor from becoming overly dominant.

[0089] Because different clients have different data and gradients, even neighboring clients may have conflicting gradients during aggregation (e.g., significant deviations in gradient direction, which can negatively impact model updates). By performing conflict detection and obtaining a set of conflicting edges, we can identify which clients may have issues with gradient aggregation, providing a basis for subsequent gradient correction.

[0090] S5, based on the set of conflicting edges, perform weighted orthogonal projection on the nearest neighbor direction frozen by each client to correct the gradient.

[0091] Specifically, the expression for the weighted orthogonal projection is:

[0092] ;

[0093] in, In the t-th round of federated learning, the client... The gradient vector; Indicates the weighted orthogonal projection , that is, the corrected gradient vector; The projection intensity coefficient; In the t-th round of federated learning, the client... The conflict edge; For the client , The weights of the conflicting edges; This represents the normalized gradient vector after freezing in the nearest neighbor direction; In the t-th round of federated learning, the client... The normalized gradient vector; Represents a very small number; express and Similarity in direction.

[0094] In this embodiment, weighted orthogonal projection is an effective method for adjusting gradients. Performing this operation on the nearest neighbor directions frozen by the client, based on the set of conflicting edges, can reconcile conflicting gradients to some extent, reducing the negative impact of conflict on model updates. For example, by projecting conflicting gradients onto nearest neighbor directions, their directions can be made more consistent, thus promoting more stable convergence of the model.

[0095] This operation weakens the conflict component while preserving the non-conflict direction.

[0096] S6 updates the global model parameters after weighted aggregation of the corrected gradients.

[0097] After correcting the gradients, weighted aggregation is a standard operation in federated learning for updating the global model parameters. The weighted average of the corrected gradients is expressed as:

[0098] ;

[0099] in, The global gradient after weighted aggregation in the t-th round of federated learning; The weight of the client; K represents the total number of clients; This represents the corrected gradient vector.

[0100] By using weighted aggregation, the contributions of different clients can be comprehensively considered. Combined with the gradient processing in the previous steps, the updated global model parameters are more reasonable, which helps to improve the generalization performance of the model on different client data.

[0101] Furthermore, the expression for updating the global model parameters is:

[0102] ;

[0103] in, , These are the global model parameters for the (t+1)th and tth rounds of federated learning, respectively; This is the learning rate.

[0104] In practical applications, the method of this invention can significantly improve the convergence speed, stability, and scalability of models in large-scale federated learning environments. For example, in a federated learning system with 1000 clients, the client data distribution is highly heterogeneous and communication bandwidth is limited. The server-side, using the method of this invention, utilizes client similarity to limit the conflict handling scope to the nearest neighbor set, avoiding the high computational complexity problem caused by global conflict handling in traditional methods. The method of this invention can significantly improve the model convergence speed and enhance the accuracy and robustness of the global model without increasing the additional computational and communication overhead of the clients.

[0105] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0106] First, this invention enhances the stability of the client-side orientation representation by updating the orientation embedding through exponential moving average, thereby improving the accuracy of conflict detection. Second, the nearest neighbor projection mechanism removes conflict components while preserving useful gradient information to the maximum extent, avoiding the model performance loss caused by excessive shaving in traditional methods. Third, by dynamically adjusting the projection intensity coefficient and weight cap, this invention can adaptively adapt to the needs of different training stages, further improving the optimization effect.

[0107] This invention proposes a novel gradient coordination method by introducing client similarity and nearest neighbor projection mechanisms, addressing the shortcomings of existing techniques in handling gradient conflicts. This method not only improves the convergence efficiency and stability of federated learning systems but also possesses good scalability and practicality, providing important technical support for the field of distributed machine learning.

[0108] Example 2

[0109] like Figure 2 As shown, the second embodiment of the present invention also provides a nearest-neighbor projection gradient coordinated compression device, comprising:

[0110] The orientation normalization module is used to obtain the gradient vector uploaded by each client and perform orientation normalization.

[0111] The exponential sliding update module is used to update the orientation embedding of each client based on the normalized gradient vector by exponential sliding average.

[0112] The nearest neighbor set generation module is used to calculate the similarity between clients based on the directional embedding and select the most similar nearest neighbor clients to generate a nearest neighbor set;

[0113] The conflict detection module is used to perform conflict detection based on similarity and the nearest neighbor set to obtain a set of conflicting edges;

[0114] The gradient correction module is used to perform a weighted orthogonal projection on the nearest neighbor directions frozen by each client to correct the gradient based on the set of conflicting edges;

[0115] The aggregation update module is used to update the global model parameters after weighted aggregation of the corrected gradients.

[0116] Example 3

[0117] The third embodiment of the present invention also provides a federated learning system, including multiple clients and a server. Each client completes local model training and updates and uploads gradient vectors to the server. The server includes a processor and a memory. The memory stores a computer program that can be executed by the processor to implement the nearest-neighbor projection gradient coordination compression method as described above.

[0118] Example 4

[0119] The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device where the computer-readable storage medium is located, implement the nearest-neighbor projection gradient coordinated compression method as described above.

[0120] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0121] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0122] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0124] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0125] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0126] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0127] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A neighborly projected gradient coordinated compression method applied to a server end of a federated learning system, characterized in that, include: S1, obtain the gradient vector uploaded by each client and normalize its direction; S2, based on the normalized gradient vector, updates the orientation embedding of each client through exponential moving average; S3, calculate the similarity between clients based on the directional embedding and select several nearest neighbor clients with the highest similarity to generate a nearest neighbor set; S4, perform conflict detection based on similarity and the nearest neighbor set to obtain a set of conflict edges; S5, based on the set of conflicting edges, perform weighted orthogonal projection on the nearest neighbor direction frozen by each client to correct the gradient; S6, after weighted aggregation of the corrected gradients, update the global model parameters; Specifically, the conflict detection includes: Based on the similarity between clients and the nearest neighbor set, if the similarity is less than a negative threshold, it is determined to be a conflict edge, and the formula is as follows: ； wherein, denotes the conflicting edge of client in the t-th round of federated learning; denotes the neighbor set of client ; denotes the direction cosine similarity of client , and client in the t-th round of federated learning; is a similarity threshold value; Next, weights are assigned to the conflicting edges: ； wherein, is the client , weight of the conflicting edge consisting of the client is the weight upper bound used to avoid single neighbor over domination; The expression for the weighted orthogonal projection is: ； wherein, denotes the gradient vector of the client in the t-th round of federated learning; denotes the gradient vector after weighted orthogonal projection, i.e. the corrected gradient vector; is the projection intensity coefficient; denotes the conflicting edge of the client in the t-th round of federated learning; denotes the normalized gradient vector after neighbor direction freezing, wherein, ; denotes the normalized gradient vector of the client in the t-th round of federated learning; denotes the minimal number; denotes the directional cosine similarity of the two; denotes the Euclidean norm.

2. The method of claim 1, wherein The formula for normalizing the gradient vector by direction is: ； wherein, denotes the normalized gradient vector of client in round t of federated learning; denotes the gradient vector of client in round t of federated learning; denotes the epsilon, to prevent division by zero; denotes the Euclidean norm of , which is used to measure the length of the gradient vector; denotes the Euclidean norm.

3. The method of claim 1, wherein the projection is a nearest neighbor projection. The expression for updating the orientation embedding for each client using exponential moving average is: ； And embed the orientation into a unit sphere: ； wherein, denotes the direction embedding of client in the t-th round of federated learning; denotes the direction embedding of client in the previous round of federated learning; denotes the normalized gradient vector of client in the t-th round of federated learning; is a smoothing coefficient; denotes a minimum number; denotes the Euclidean norm.

4. The method of claim 1, wherein The similarity between clients is obtained by calculating the cosine similarity of the directional embeddings of each pair of clients, expressed as: ； wherein, denotes the direction cosine similarity of the client , in the t-th round of federated learning; , denotes the direction embedding of the client in the t-th round of federated learning; is a neighbor of the client .

5. The nearest-neighbor projection gradient coordinated compression method according to claim 1, characterized in that... The weighted average of the corrected gradients is expressed as follows: ； in, The global gradient after weighted aggregation in the t-th round of federated learning; The weight of the client; K represents the total number of clients; This represents the corrected gradient vector.

6. The nearest-neighbor projection gradient coordinated compression method according to claim 5, characterized in that... The expression for updating the global model parameters is: ； in, , These are the global model parameters for the (t+1)th and tth rounds of federated learning, respectively; This is the learning rate.

7. A nearest-neighbor projection gradient coordination compression device, applied to the server side of a federated learning system, to implement the nearest-neighbor projection gradient coordination compression method as described in any one of claims 1-6, characterized in that, include: The orientation normalization module is used to obtain the gradient vector uploaded by each client and perform orientation normalization. The exponential sliding update module is used to update the orientation embedding of each client based on the normalized gradient vector by exponential sliding average. The nearest neighbor set generation module is used to calculate the similarity between clients based on the directional embedding and select the most similar nearest neighbor clients to generate a nearest neighbor set; The conflict detection module is used to perform conflict detection based on similarity and the nearest neighbor set to obtain a set of conflicting edges; The gradient correction module is used to perform a weighted orthogonal projection on the nearest neighbor directions frozen by each client to correct the gradient based on the set of conflicting edges; The aggregation update module is used to update the global model parameters after weighted aggregation of the corrected gradients.

8. A federated learning system comprising multiple clients and a server, each client completing local model training and updating and uploading gradient vectors to the server; the server comprising a processor and a memory, the memory storing a computer program that can be executed by the processor to implement a nearest-neighbor projection gradient coordination compression method as described in any one of claims 1-6.

Images