Federated learning architecture under dynamic bandwidth and unreliable network and compression algorithm of architecture

Abstract

The invention provides a federated learning architecture under a dynamic bandwidth and unreliable network and a compression algorithm of the architecture. The architecture comprises a cloud, a clientand a plurality of edge servers, the edge servers can exchange data with each other, and one of the edge servers is used as a coordinator. Data transmission between the client and the edge servers istransmission of data blocks, and an adopted edge server mechanism shortens a traditional transmission distance between the client and the cloud, and the communication delay between the client and theserver is reduced. Besides, a plurality of edge servers are introduced, and a trained model is divided into corresponding number of blocks, so that influence of data packet loss on training is reducedby avoiding losing the whole model. The compression algorithm adaptively adjusts the compression rate to adapt to the dynamic bandwidth, and the bandwidth is adaptively adjusted to replace the fixedcompression rate, so that bandwidth resources are effectively utilized, and redundant data exchange is reduced.

Description

technical field [0001] The invention relates to the field of federated learning, and specifically discloses a federated learning framework and a compression algorithm thereof under dynamic bandwidth and unreliable networks. Background technique [0002] Emerging issues such as privacy protection and lifelong learning bring many new challenges to big data and artificial intelligence, such as information leakage, legal violations, model mismatch, and resource constraints. In addition, the General Data Protection Regulation (GDPR) and other relevant laws further restrict the use of such data. These factors create isolated data islands in the network, which makes the current big data less bulky. At the same time, models trained by general datasets cannot adapt to users' personalized requirements (such as words outside the vocabulary) and social changes (such as Internet vocabulary, new terms, etc.). In order to meet these requirements, AI must be able to relearn from the lates...

AI Technical Summary

Problems solved by technology

However, such approaches do not address many of the challenges of communication constraints, including: Unreliable networks: Since many factors can affect communication, and as traffic increases, so does the likelihood of communication interruptions

Additionally, participating customers (including smartphones, laptops, self-driving cars, etc.) may stop transmitting due to movement, battery drain, or other reasons

However, most existing federated learning works have not studied this challenge deeply; Network throughput: In traditional federated learning algorithms, all selected clients send their local models to the same server

These clients are usually large in scale, which will lead to bandwidth peak occupation and impose strict requirements on network throughput

In addition, the carrying capacity of the network also limits the scale of the client, which may cause the newly generated data to fail to operate normally; dynamic bandwidth: under dynamic bandwidth, federated learning must extend its global sharing time to wait for those clients with lower bandwidth

This slows down training and fusion

In addition, the asymmetry of Internet connection speed (for example, in 2019, the US mobile download speed is 33.88 Mbps, while the upload speed is 9.75 Mbps) also wastes a lot of bandwidth resources; theoretical convergence: the data distribution of federated learning is unbalanced and Non-IID (non-independent and identical), while most distributed learning methods assume an IID distribution

Furthermore, most federated learning algorithms are only validated experiments, and the impact of unreliable networks and communication compression is rarely analyzed theoretically

Images