Federated learning architecture under dynamic bandwidth and unreliable network and compression algorithm of architecture

A dynamic bandwidth and compression algorithm technology, applied in machine learning, data exchange network, computing, etc., can solve problems such as limiting client scale, increasing communication interruption, bandwidth peak occupation, etc., to reduce redundant data exchange and communication delay , the effect of using bandwidth resources

Active Publication Date: 2020-07-24
NAT UNIV OF DEFENSE TECH
View PDF4 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, such approaches do not address many of the challenges of communication constraints, including: Unreliable networks: Since many factors can affect communication, and as traffic increases, so does the likelihood of communication interruptions
Additionally, participating customers (including smartphones, laptops, self-driving cars, etc.) may stop transmitting due to movement, battery drain, or other reasons
However, most existing federated learning works have not studied this challenge deeply; Network throughput: In traditional federated learning algorithms, all selected clients send their local models to the same server
These clients are usually large in scale, which will lead to bandwidth peak occupation and impose strict requirements on network throughput
In addition, the carrying capacity of the network also limits the scale

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Federated learning architecture under dynamic bandwidth and unreliable network and compression algorithm of architecture
  • Federated learning architecture under dynamic bandwidth and unreliable network and compression algorithm of architecture
  • Federated learning architecture under dynamic bandwidth and unreliable network and compression algorithm of architecture

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] Federated learning is a special form of federated optimization for machine learning. Therefore, this embodiment first defines federated optimization as a type of distributed optimization with a data-parallel setting, where data is distributed across different computing devices, as follows:

[0048]

[0049] where f is the global optimization objective, f i :R d → R is the target defined by the local data available on each node, w i ∈ R d means f i solution.

[0050] When solving a federated optimization problem, each computing device searches f from its local data i the solution w i , then the server uses the aggregation algorithm f agg will w i Aggregated to obtain the global solution w * :

[0051] w * = f agg (w 1 ,...w n ) (2)

[0052] The server then takes the global solution w * as its new w i distributed to each client. Each customer starts with this new w i Search for a better solution for a starting point. The client and server repeat the...

Embodiment 2

[0143] This embodiment uses image classification, sentiment analysis and next character prediction tasks as experimental examples to evaluate the effectiveness of ACFL. All of these tasks correspond to typical machine learning models including Convolutional Neural Networks (CNN), Bag-of-Words Logistic Regression (Bag-Log-Reg), and Long Short-Term Memory (LSTM). The datasets for these tasks conform to federated settings characterized by non-IDD, imbalance, and large-scale distribution. For the image classification task, this example uses Federated Extended MNIST (FEMNIST), which is constructed from the author's partitioning of data in Extended MNIST. There are 62 different categories (10 digits, 26 lowercase letters, 26 uppercase letters) images in FEMNIST, 28 x 28 pixels with 3500 users. For the sentiment analysis task, this example uses Sentiment140, which is constructed by annotating tweets according to the emoji present in them and partitioning them according to 660120 Twi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a federated learning architecture under a dynamic bandwidth and unreliable network and a compression algorithm of the architecture. The architecture comprises a cloud, a clientand a plurality of edge servers, the edge servers can exchange data with each other, and one of the edge servers is used as a coordinator. Data transmission between the client and the edge servers istransmission of data blocks, and an adopted edge server mechanism shortens a traditional transmission distance between the client and the cloud, and the communication delay between the client and theserver is reduced. Besides, a plurality of edge servers are introduced, and a trained model is divided into corresponding number of blocks, so that influence of data packet loss on training is reducedby avoiding losing the whole model. The compression algorithm adaptively adjusts the compression rate to adapt to the dynamic bandwidth, and the bandwidth is adaptively adjusted to replace the fixedcompression rate, so that bandwidth resources are effectively utilized, and redundant data exchange is reduced.

Description

technical field [0001] The invention relates to the field of federated learning, and specifically discloses a federated learning framework and a compression algorithm thereof under dynamic bandwidth and unreliable networks. Background technique [0002] Emerging issues such as privacy protection and lifelong learning bring many new challenges to big data and artificial intelligence, such as information leakage, legal violations, model mismatch, and resource constraints. In addition, the General Data Protection Regulation (GDPR) and other relevant laws further restrict the use of such data. These factors create isolated data islands in the network, which makes the current big data less bulky. At the same time, models trained by general datasets cannot adapt to users' personalized requirements (such as words outside the vocabulary) and social changes (such as Internet vocabulary, new terms, etc.). In order to meet these requirements, AI must be able to relearn from the lates...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L12/24H04L29/06H04L29/08G06N20/00
CPCH04L41/0896H04L67/10H04L69/04G06N20/00Y02D30/50
Inventor 朱晓敏张雄涛包卫东梁文谦周文司悦航王吉吴冠霖陈超高雄闫辉张亮
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap