Methods, equipment, and products for building an efficient and scalable federated learning system based on P2P

Abstract

The present invention discloses a method, device and product for constructing an efficient and scalable federated learning system based on P2P. First, the parameter server selects the client based on historical information such as statistical efficiency, processing speed, and distribution contribution, and the client allocator assigns a dedicated executor to the selected client. The executor runs in an isolated container environment, conducts real communication through a dedicated network card, and distributes the global model using a P2P communication library; the executor sends the training configuration, local data set, and global model to the computing node, and the computing node returns the model update to the executor after several rounds of local training; the parameter server receives the model update from the executor and generates a new global model. The present invention optimizes model distribution through P2P communication and designs a client selection scheme that comprehensively considers multiple historical information. In addition, FedP2P builds a high-fidelity simulation platform to accurately simulate the computing and communication behaviors in actual federated training, thereby verifying the effectiveness of the scheme design.

Description

Technical Field

[0001] The present invention belongs to the field of computer network communication and distributed machine learning technology, and relates to a method, device and product for constructing a federated learning system, and specifically to a method, device and product for constructing a system based on P2P (peer-to-peer) communication and cross-device federated learning. Background Art

[0002] Machine learning has achieved remarkable success in many fields, including translation and speech-to-text conversion. Training a high-performance model typically requires massive amounts of user data. However, due to growing concerns about data privacy in recent years, model developers cannot directly collect user data. Federated learning (FL), a new distributed training paradigm, enables tens of thousands of clients to collaboratively train a global model without leaking local private data, effectively addressing data privacy and security issues in distributed training.

[0003] In traditional federated learning (FL) systems, the parameter server (PS) is responsible for coordinating the clients participating in training, including client selection, model distribution, and model aggregation. However, in real-world FL, the number of clients can often reach hundreds or even tens of thousands. Due to the limited bandwidth of the PS, communication between the PS and a large number of client devices over slow wireless networks can become a major performance bottleneck for the system. Existing solutions to this bottleneck primarily rely on model distribution based on the existing one-to-many communication model, employing algorithmic optimizations such as momentum-based model updates and improved federated learning aggregation algorithms, without focusing on the communication itself.

[0004] As a distributed communication protocol, P2P not only allows communication between PSs and clients, but also allows clients to communicate directly with each other. This means that clients can send received model blocks to other clients, thereby reducing the communication burden on PSs and accelerating the model distribution process. Although P2P architectures have been applied to FL to improve communication efficiency, existing solutions mainly focus on decentralized model aggregation, removing PSs and allowing clients to communicate and aggregate models directly. Furthermore, these designs do not change the underlying communication primitives.

[0005] While the technical path to accelerating cross-device federated learning training through P2P communication is clear, implementation presents numerous challenges. First, existing P2P communication protocols, such as BitTorrent, are designed to ensure fair file transfer between clients. However, these solutions only aim to maximize the efficiency of model distribution in federated learning and do not address fairness. Furthermore, to improve training efficiency, these solutions require that all read and write operations avoid slow devices like hard drives. Therefore, they require porting existing protocols and implementing memory management to ensure that all read and write operations are completed within high-speed memory. Second, existing client selection schemes require collecting historical information on client efficiency and processing speed. Because they are all based on a one-to-many communication architecture, adopting P2P communication also requires additional consideration of the client distribution process, selecting clients that contribute significantly to the distribution process. Third, training on hundreds or even tens of thousands of devices typically incurs significant overhead, necessitating the development of a simulation platform that can simulate hundreds or tens of thousands of devices. Existing FL simulation platforms typically calculate communication times rather than measure them. Furthermore, they lack support for simulated communication, making it difficult to validate new communication architectures on these platforms.

[0006] Therefore, how to accelerate federated learning training through P2P communication, how to design new client selection schemes, and how to build a simulation platform to verify the effectiveness of these designs are challenges that need to be solved urgently. Summary of the Invention

[0007] This paper aims to address three challenges in the existing technology: (1) How to design a P2P communication backend so that the idle bandwidth of the client can be fully utilized during the distribution process and ultimately accelerate FL training. (2) How to design a client selection scheme so that the client's statistical efficiency, processing speed, distribution contribution and other information can be comprehensively considered to improve the system's training efficiency. (3) How to design a FL simulation platform so that clients can communicate with each other in a realistic manner and ensure the authenticity of the simulation platform's operation results.

[0008] The technical solution adopted by the method of the present invention is: a method for constructing an efficient and scalable federated learning system based on P2P, wherein the global model of the efficient and scalable federated learning system is obtained through training, and the training process includes the following steps:

[0009] Step 1: The parameter server PS selects the client based on historical information, including statistical efficiency, processing speed, and distribution contribution;

[0010] Step 2: Each selected client designates a dedicated executor for P2P communication, sending the client's training configuration to the computing node; the executor runs in an isolated container environment and is equipped with a dedicated network card for real communication;

[0011] Step 4: The executor sends the training configuration, local dataset, and global model to the compute node. After the compute node performs several rounds of local training, it sends the global model update to the executor.

[0012] Step 5: The parameter server PS receives global model updates from all executors;

[0013] Step 6: The parameter server PS generates a new global model based on the received global model update.

[0014] Preferably, in step 1, a guided client selection algorithm is used to select a client by comprehensively considering three factors: statistical performance, processing speed, and P2P contribution of the client;

[0015] The guided client selection algorithm, for the clients selected in the previous round, the parameter server PS collects the time required for them to complete training and the parameter gradient information, and completes the exploration of the client set. Update of statistical performance set U, last selected round set L, and current round time set D; for each explored client According to its statistical performance U(i), the last selected round L(i), processing speed D(i) and upload contribution u i Calculate its utility Among them, R represents the current round; then the clients are sorted in descending order according to the utility Util, and (1-∈)×N clients are selected and put into S according to the probability generated by Util t Among them, the client selected in the tth round is recorded as S t , N is the number of clients, ∈ represents the exploration factor, randomly select ∈×N clients from the client set that has never been explored and put them into S t middle.

[0016] Preferably, the specific implementation of step 3 includes the following sub-steps:

[0017] Step 3.1: The P2P communication library selects a predetermined number of relatively scarce model blocks from all model blocks;

[0018] Step 3.2: The P2P communication library selects the peer with the best service capability from all the peers that have established connections;

[0019] Step 3.3: The client requests the model nugget from the selected peer.

[0020] Step 3.4: The peer transmits the model block to the client.

[0021] Preferably, in step 3.1, the P2P communication library selects a relatively scarce model block from all model blocks according to a random rare-first model block selection algorithm;

[0022] The random rarity-first model block selection algorithm first evaluates the rarity of model block j in client i r(i,j)=∑ k∈c(i) (1-P(k,j)), where client i has established a connection with peers k∈c(i), and the set of peers to which client i has established a connection is c(i); whether client i owns model block j, P(i,j)=1 indicates yes, and P(i,j)=0 indicates no; the higher r(i,j), the rarer model block j is in client i's network; then, all unowned blocks are grouped according to rarity r(i,j), and within each group, the block indices are randomly shuffled; finally, the model blocks in each group are merged in descending order of rarity to form the final block request order.

[0023] Preferably, the P2P communication library selects the peer with the best service capability from all peers with which connections have been established according to an active peer selection algorithm;

[0024] The peer selection algorithm checks all peers k∈c(i) that have established connections with client i. If peer k has the model block j required by peer i and the maximum number of requests initiated by peer i to peer k does not exceed the limit, peer i will request model block j from peer k; at the same time, the window mechanism in TCP is used to dynamically adjust the maximum number of requests, that is, if peer k completes the request target, the window will increase, and if not, the window will be reduced to a non-zero value.

[0025] Preferably, the federated learning system introduces a computing node architecture based on the executor, where each computing node is responsible for computing tasks of multiple clients on the same physical host. Each computing node allocates GPU resources on demand to ensure efficient use of the GPU.

[0026] Preferably, the federated learning system adopts a synchronization barrier mechanism. At the end of each round of training, all clients must complete the model update before the aggregation step, ensuring that the computing and communication time of all clients are effectively synchronized, avoiding time misalignment problems caused by executor resource limitations.

[0027] The present invention also provides a device for constructing an efficient and scalable federated learning system based on P2P, comprising:

[0028] one or more processors;

[0029] A storage device is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the method for constructing an efficient and scalable P2P-based federated learning system.

[0030] The present invention also provides a P2P-based efficient and scalable federated learning system construction product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the P2P-based efficient and scalable federated learning system construction method.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) This paper proposes an efficient and scalable federated learning system based on P2P, FedP2P. It applies the peer-to-peer (P2P) communication protocol to the model distribution stage of federated learning for the first time and adaptively optimizes the client selection strategy, effectively alleviating the communication bottleneck problem at the parameter server (PS) in the traditional federated learning model distribution stage and improving the model training efficiency.

[0033] (2) FedP2P has developed a dedicated P2P communication backend. Through the randomized rarity-first model block selection strategy and the active peer selection strategy, it fully utilizes the client's upload and download bandwidth in the model distribution phase, significantly eliminating the communication bottleneck of the PS in the model distribution phase and speeding up the model distribution.

[0034] (3) FedP2P designs a guided client selection algorithm that comprehensively considers the client's statistical performance, processing speed, and contribution to P2P distribution, guiding the selection of the optimal participants in subsequent rounds and improving the training efficiency of federated learning.

[0035] (4) FedP2P builds a high-fidelity simulation platform that accurately simulates the real federated learning training process, supports accurate client communication and runtime computation calibration, and precisely replicates the parallel computing and communication behaviors of large-scale devices, ensuring the reliability and authenticity of the simulation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The technical solution of the present invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without making any creative efforts.

[0037] Figure 1Schematic diagram of the method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0039] Please see Figure 1 This embodiment provides an efficient and scalable method for constructing a federated learning system based on P2P. First, the formal definition of the symbol of the federated learning system (FedP2P) in this example is given:

[0040] Client Collection

[0041] Client parameters

[0042] The client's local dataset

[0043] Number of local samples on the client

[0044] Aggregate weight of the client

[0045] Client S selected in round t t ;

[0046] The total number of training rounds T, the number of local training rounds E on the client side, the learning rate η, and the global model w at round t t ;

[0047] Client i has established a connected peer c(i);

[0048] Whether client i has model block j, P(i,j);

[0049] The rarity assigned by client i to model block j, r(i,j);

[0050] The global model of the efficient and scalable federated learning system is obtained through training. The training process includes the following steps:

[0051] Step 1: The parameter server PS selects the client based on historical information, including statistical efficiency, processing speed, and distribution contribution;

[0052] Traditional federated learning client selection methods are mainly based on the client's statistical performance (such as the representativeness of data distribution) or processing speed (such as local computing power), but fail to fully consider the client's contribution to P2P model distribution. This single-dimensional selection strategy may lead to the following problems: (1) Inefficiency. Failure to consider P2P contribution may fail to fully utilize the client's upload bandwidth, limiting the speed of model propagation. (2) System instability: If the selected client has a low contribution in P2P propagation, it may become a laggard, extending the overall training time.

[0053] In one implementation, the guided client selection algorithm proposed by FedP2P comprehensively considers the three factors of client statistical performance, processing speed, and P2P contribution, making it more suitable for FedP2P systems that accelerate model distribution through P2P. Specifically, for the clients selected in the previous round, PS collects information such as the time required for them to complete training, parameter gradients, etc., and completes the exploration of the client set. Updates to the statistical performance U, the last selected round L, the current round time D, etc. For each explored client According to its statistical performance U(i), the last selected round L(i), processing speed D(i) and upload contribution u i Calculate its utility Among them, R represents the current round. Then, the clients are sorted in descending order according to the utility Util, and (1-∈)×N clients are selected and put into S according to the probability generated by Util. t In order to avoid long-term dependence on a fixed client subset and improve the robustness and adaptability of the system, the algorithm adds an exploration factor ∈, randomly selecting ∈×N clients from the client set that has never been explored and putting them into S t middle.

[0054]

[0055]

[0056] In the client selection algorithm described above, α is a penalty factor that penalizes clients whose processing time exceeds the expected time. β is a factor that rewards clients whose upload contribution exceeds the model file size, or penalizes clients whose upload contribution is less than the model file size. F represents the size of the model file. c is a decimal less than 1, representing the confidence factor used to set the utility threshold in the participant selection process. For example, c = 0.95 means that the algorithm will select participants whose utility reaches at least 95% of the utility value of the (1-∈)×Kth participants.

[0057] Step 2: Each selected client designates a dedicated executor for P2P communication, sending the client's training configuration to the computing node; the executor runs in an isolated container environment and is equipped with a dedicated network card for real communication;

[0058] Cross-device federated learning involves parallel computing and communication across a large number of clients. Deploying thousands or even millions of clients in practice is extremely difficult and resource-intensive. Therefore, a simulation platform that can realistically simulate the complex communication and computing behaviors in a federated learning environment is crucial for verifying the effectiveness and scalability of FedP2P.

[0059] In one embodiment, FedP2P constructs a simulation platform that can realistically simulate the complex communication and computing behaviors in a federated learning environment from three aspects: communication, GPU utilization, and simulation reliability. (1) Communication. FedP2P uses containerization technology to allocate an independent operating environment (executor) to each client. Each container has an independent network interface and can simulate the network behavior of a real client. This method is not only lightweight, but also allows a large number of client instances to run on a single physical machine, greatly improving the scalability of the simulation platform. FedP2P uses Linux traffic control tools to accurately configure the network bandwidth and latency of each client at runtime, so that the simulation platform can simulate the heterogeneous network conditions of different clients and ensure the authenticity of communication behavior. (2) GPU utilization. FedP2P introduces a computing node architecture based on the executor. Each computing node is responsible for the computing tasks of multiple clients on the same physical host, avoiding frequent switching of GPU resources between multiple executors and reducing the repeated occupation of GPU memory. Each computing node allocates GPU resources on demand to ensure efficient utilization of the GPU. (3) Simulation reliability. Simulation reliability requires ensuring the accuracy of communication time, computing time, and the alignment of the two. First, FedP2P uses a containerized runtime environment and network configuration control to realistically simulate parallel communication between clients, ensuring accurate communication time. In simulation, actual clients do not perform actual local training tasks. Instead, computation time is estimated based on the client's computing power, local data volume, and real-world computing datasets, ensuring that the simulated computation time is highly consistent with the actual situation. To accurately simulate the synchronization behavior during parallel training, FedP2P introduces a synchronization barrier mechanism. At the end of each training round, all clients must complete the model update before the aggregation step, ensuring that the computation and communication times of all clients are effectively synchronized, avoiding time misalignment issues caused by executor resource constraints.

[0060] Step 3: The executor uses the P2P communication library to complete the global model distribution;

[0061] In one embodiment, the specific implementation includes the following sub-steps:

[0062] Step 3.1: Based on the Randomized Rarity-First (RRF) model block selection algorithm, the P2P communication library selects a certain number of relatively scarce model blocks from all model blocks;

[0063] RRF aims to improve the efficiency of model distribution and the scalability of the system in the FedP2P system. Compared with the traditional Rarity-First (RF) model block selection algorithm, RRF introduces a randomization factor into the model blocks of the same rarity, making the requests of different client nodes more diverse, avoiding clients with scarce model blocks from being overwhelmed by requests from a large number of nodes, and also facilitating the diffusion of model blocks among clients, thereby making more effective use of the client's uplink and downlink bandwidth. Specifically, the algorithm first evaluates the rarity of model block j in client i, r(i,j)=∑ k∈c(i) (1-P(k,j)), where k∈c(i) is the number of peers to which client i has established a connection, and c(i) is the set of peers to which client i has established a connection. Whether client i possesses model block j is indicated by P(i,j)=1, and P(i,j)=0. The higher r(i,j), the rarer model block j is in client i's network. Subsequently, all unowned blocks are grouped by rarity r(i,j). Within each group, the block indices are randomly shuffled to avoid excessively concentrating requests on certain nodes, which could lead to network congestion and reduced communication efficiency. Finally, the model blocks in each group are merged in descending rarity order to form the final block request order.

[0064]

[0065]

[0066] Step 3.2: Based on the active peer selection algorithm, the P2P communication library selects the peer with the best service capability from all the peers with which connections have been established;

[0067] Active peer selection algorithms are key technologies for improving model distribution efficiency and system scalability. Traditional BitTorrent uses a tit-for-tat and optimistic unchoking mechanism. This passive decision-making mechanism is designed to ensure fairness during transmission and prevent some nodes from only downloading and not uploading, which would affect the overall communication efficiency of the system. Federated learning, on the other hand, assumes that clients train collaboratively, and the goal of completing training quickly and efficiently is the common goal of every client. Therefore, it is necessary to remove the fairness mechanism in BitTorrent and propose an active peer selection algorithm to ensure that each peer obtains the upload service of the best node, thereby improving bandwidth utilization and overall communication efficiency. Specifically, the algorithm checks all peers k∈c(i) that have established a connection with client i. If peer k has the model block j required by peer i and the maximum number of requests initiated by peer i to peer k is within the limit, peer i will request model block j from peer k. The algorithm adopts the window mechanism in TCP to dynamically adjust the maximum number of requests. That is, if peer k completes the request target, the window will increase. If it is not completed, the window will be reduced to a non-zero value.

[0068]

[0069] Step 3.3: The client requests the model nugget from the selected peer.

[0070] Step 3.4: The peer transmits the model block to the client.

[0071] Step 4: The executor sends the training configuration, local dataset, and global model to the compute node. After the compute node performs several rounds of local training, it sends the global model update to the executor.

[0072] Step 5: The parameter server PS receives global model updates from all executors;

[0073] Step 6: The parameter server PS generates a new global model based on the received global model update.

[0074]

[0075]

[0076] This embodiment also provides a device for building an efficient and scalable federated learning system based on P2P, including:

[0077] one or more processors;

[0078] A storage device is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the method for constructing an efficient and scalable P2P-based federated learning system.

[0079] This embodiment also provides a P2P-based efficient and scalable federated learning system construction product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the P2P-based efficient and scalable federated learning system construction method.

[0080] The present invention is further described below through specific experiments.

[0081] This experiment was conducted on a GPU cluster consisting of seven physical servers, connected via 10Gbps Ethernet. Each server was equipped with 128GB of RAM and an NVIDIA GeForce RTX 4070 GPU with 12GB of video memory. To simulate heterogeneous clients, this experiment sampled real-world computational power and bandwidth datasets as the client's computational and communication capabilities. The upload and download bandwidth of the PS was configured to be 1000Mbps. Fifty executors were deployed in the cluster using Docker, corresponding to 50 participants in each round.

[0082] In terms of model and dataset selection, this experiment trained the ResNet-18 model on the FEMNIST dataset for image classification tasks, trained the ShuffleNet V2 model on the OpenImage dataset for image classification tasks, and trained the ResNet-34 model on the Google Speech dataset for speech recognition tasks.

[0083] In terms of the selection of baseline methods, this experiment compares FedAvg and Oort with the method of the present invention, calculates the time taken to reach the target test accuracy, and uses FedAvg as the baseline for comparison to calculate the speedup ratio.

[0084] In terms of experimental results, for the FEMNIST dataset, the three methods took 6.70h, 6.64h, and 4.73h, respectively, with speedups of 1.00x, 1.01x, and 1.41x, respectively. For the OpenImage dataset, the three methods took 33.92h, 27.74h, and 19.97h, respectively, with speedups of 1.00x, 1.22x, and 1.70x, respectively. For the Google Speech dataset, the three methods took 10.21h, 8.35h, and 7.40h, respectively, with speedups of 1.00x, 1.22x, and 1.38x, respectively. This shows that the method of the present invention can effectively alleviate the communication bottleneck problem at the parameter server PS in the model distribution phase of federated learning and accelerate the training process of the system.

[0085] The FedP2P system of the present invention is implemented as a Python library and is compatible with existing communication backends (such as MPI, Gloo, and gRPC), providing an easy-to-migrate, integrate, and deploy solution, enabling researchers and developers to conveniently apply and extend the functionality of FedP2P.

[0086] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0087] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A method for constructing an efficient and scalable federated learning system based on P2P, characterized in that: The global model of the efficient and scalable federated learning system is obtained through training. The training process includes the following steps: Step 1: The parameter server PS selects the client based on historical information, including statistical performance, processing speed, and P2P contribution; Step 2: Each selected client designates a dedicated executor for P2P communication, sending the client's training configuration to the computing node; the executor runs in an isolated container environment and is equipped with a dedicated network card for real communication; Step 3: The executor uses the P2P communication library to complete the global model distribution; Step 4: The executor sends the training configuration, local dataset, and global model to the compute node. After the compute node performs several rounds of local training, it sends the global model update to the executor. Step 5: The parameter server PS receives global model updates from all executors; Step 6: The parameter server PS generates a new global model based on the received global model update.

2. The method for constructing an efficient and scalable P2P-based federated learning system according to claim 1, characterized in that: In step 1, a guided client selection algorithm is used to select clients by comprehensively considering three factors: statistical performance, processing speed, and P2P contribution of the client. The guided client selection algorithm, for the clients selected in the previous round, the parameter server PS collects the time required for them to complete training and the parameter gradient information, and completes the exploration of the client set. Update of statistical performance set U, last selected round set L, and current round time set D; for each explored client According to its statistical performance U(i), the last selected round L(i), the processing speed D(i) and the P2P contribution u i Calculate its utility Among them, R represents the current round; then the clients are sorted in descending order according to the utility Util, and (1-∈)×N clients are selected and put into S according to the probability generated by Util t Among them, the client selected in the tth round is recorded as S t , N is the number of clients, ∈ represents the exploration factor, randomly select ∈×N clients from the client set that has never been explored and put them into S t middle.

3. The method for constructing an efficient and scalable P2P-based federated learning system according to claim 1, characterized in that: The specific implementation of step 3 includes the following sub-steps: Step 3.1: The P2P communication library selects a predetermined number of relatively scarce model blocks from all model blocks; Step 3.2: The P2P communication library selects the peer with the best service capability from all the peers that have established connections; Step 3.3: The client requests the specified model block from the selected peer; Step 3.4: The peer transmits the model block to the client.

4. The method for constructing an efficient and scalable P2P-based federated learning system according to claim 3, characterized in that: In step 3.1, the P2P communication library selects the relatively scarce model blocks from all model blocks according to the random rare-first model block selection algorithm; The random rarity-first model block selection algorithm first evaluates the rarity of model block j in client i r(i,j)=∑ k∈c(i) (1-P(k,j)), where client i has established a connection with peers k∈c(i), and the set of peers to which client i has established a connection is c(i); whether client i owns model block j, P(i,j)=1 indicates yes, and P(i,j)=0 indicates no; the higher r(i,j), the rarer model block j is in client i's network; then, all unowned blocks are grouped according to rarity r(i,j), and within each group, the block indices are randomly shuffled; finally, the model blocks in each group are merged in descending order of rarity to form the final block request order.

5. The method for constructing an efficient and scalable P2P-based federated learning system according to claim 3, characterized in that: In step 3.2, based on the active peer selection algorithm, the P2P communication library selects the peer with the best service capability from all the peers with which connections have been established; The peer selection algorithm checks all peers k∈c(i) that have established connections with client i. If peer k has the model block j required by peer i and the maximum number of requests initiated by peer i to peer k does not exceed the limit, peer i will request model block j from peer k; at the same time, the window mechanism in TCP is used to dynamically adjust the maximum number of requests, that is, if peer k completes the request target, the window will increase, and if not, the window will be reduced to a non-zero value.

6. The method for constructing an efficient and scalable P2P-based federated learning system according to any one of claims 1 to 5, characterized in that: The federated learning system introduces a computing node architecture based on the executor. Each computing node is responsible for computing tasks of multiple clients on the same physical host. Each computing node allocates GPU resources on demand to ensure efficient use of the GPU.

7. The method for constructing an efficient and scalable P2P-based federated learning system according to any one of claims 1 to 5, characterized in that: The federated learning system adopts a synchronization barrier mechanism. At the end of each round of training, all clients must complete the model update before the aggregation step, ensuring that the computing and communication time of all clients are effectively synchronized, avoiding the time misalignment problem caused by executor resource limitations.

8. A device for building an efficient and scalable federated learning system based on P2P, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the P2P-based efficient and scalable federated learning system construction method as described in any one of claims 1 to 7.

9. A product for building an efficient and scalable federated learning system based on P2P, including computer program instructions, characterized by: When the computer program instructions are executed on a computer, the computer is caused to execute the method for constructing an efficient and scalable federated learning system based on P2P as described in any one of claims 1 to 7.

Images