Distributed machine learning method capable of tolerating untrusted nodes

A machine learning and distributed technology, applied in machine learning, integrated learning, instruments, etc., can solve problems such as weak working nodes, difficulty in ensuring the credibility of working nodes, untrustworthy working nodes, etc., and achieve the effect of improving robustness

Pending Publication Date: 2020-07-03
NANJING UNIV
View PDF2 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this case the worker nodes may be untrusted
In addition, in applications such as edge computing and federated learning, the server organizer has weak control over the worker nodes, making it difficult t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed machine learning method capable of tolerating untrusted nodes
  • Distributed machine learning method capable of tolerating untrusted nodes
  • Distributed machine learning method capable of tolerating untrusted nodes

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] Below in conjunction with specific embodiments, the present invention will be further illustrated, and it should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. The modifications all fall within the scope defined by the appended claims of this application.

[0035] The distributed machine learning method that can tolerate untrusted nodes provided by the present invention can be applied not only to multi-machine cluster distributed machine learning, but also to edge computing, federated learning and other applications, and is suitable for a large number of data sets to be classified. , The scene where the machine learning model used has a large number of parameters is also suitable for the scene where the data is distributed on each terminal, but the training data cannot be sent due to various reasons. The present invention can be applied to various tasks such as image classificat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed machine learning method capable of tolerating untrusted nodes, which comprises the following steps that: each working node acquires latest parameters from a server node, calculates a gradient according to locally stored data, sends the gradient to the server node, and repeats the step until a stop message of a server is received. The server node is provided with a certain number of buffers, after gradient information is received each time, corresponding buffer numbers are calculated according to the numbers of the sender working nodes, and values in the buffers are updated to be average values of all gradients received corresponding to the buffers; judging whether all buffers have gradients or not, if yes, calculating a final gradient according to thegradients in all the buffers through an aggregation function, updating model parameters, and emptying all the buffers; the latest parameters are sent to the working node; and continuously repeating the training steps until a stop condition is met, and notifying each working node to stop.

Description

technical field [0001] The invention relates to a distributed machine learning method that can tolerate untrusted nodes, which can effectively reduce the negative effects brought by the erroneous gradient information of untrusted nodes in distributed machine learning, and improve the robustness of distributed machine learning. Background technique [0002] Many machine learning models can be formulated as finite and optimization problems: [0003] [0004] where w is the parameter of the model, n is the total number of training samples, ξ i represents the ith sample, f(w;ξ i ) represents the loss function corresponding to the ith sample, and d is the size of the model dimension. Stochastic Gradient Descent (SGD) and its variants are currently the most widely used methods for solving the above-mentioned finite and optimization problems. [0005] Parameter server architecture (Parameter Server) is a commonly used architecture in distributed machine learning. The paramet...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N20/00G06N20/20
CPCG06N20/00G06N20/20
Inventor 李武军杨亦锐
Owner NANJING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap