Unlock instant, AI-driven research and patent intelligence for your innovation.

Gradient information updating method for distributed deep learning and related device

A gradient information, deep learning technology, applied in the computer field, can solve problems such as reducing the efficiency of deep learning, and achieve the effect of reducing the number of communications, maintaining reliability, and reducing the amount of data

Pending Publication Date: 2020-09-04
LANGCHAO ELECTRONIC INFORMATION IND CO LTD
View PDF6 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The communication strategy in the existing technology will seriously reduce the efficiency of model training and reduce the efficiency of deep learning

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Gradient information updating method for distributed deep learning and related device
  • Gradient information updating method for distributed deep learning and related device
  • Gradient information updating method for distributed deep learning and related device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The core of this application is to provide a gradient information update method for distributed deep learning, a gradient information update device, computer equipment, and a computer-readable storage medium. Carry out coefficient communication, and at the same time, after receiving the gradient information, correct the reduced gradient to obtain the target gradient, and finally use the target gradient to update the copy model, which reduces the number of communications on the basis of sparse communication and also reduces the amount of passing data , and finally the correction process is used to maintain the reliability of gradient information, realize sparse communication with less data volume, and improve the efficiency and speed of sparse communication in distributed deep learning.

[0049]In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a gradient information updating method for distributed deep learning, and the method comprises the steps: enabling a node to carry out the sparse communication of calculated gradient information according to a preset gradient threshold value when the number of iterations is greater than a preset switching number; performing reduction processing on the received gradient information to obtain reduced gradient information; and correcting the reduced gradient information according to a set inertia coefficient to obtain a target gradient, and updating the copy model in the node according to the target gradient. Under the condition that the precision is not lost, the communication data volume is reduced, and the communication duration is shortened. The invention further discloses a gradient information updating device for distributed deep learning, computer equipment and a computer readable storage medium, which have the above beneficial effects.

Description

technical field [0001] The present application relates to the field of computer technology, and in particular to a method for updating gradient information of distributed deep learning, a device for updating gradient information, computer equipment, and a computer-readable storage medium. Background technique [0002] With the continuous development of information technology, the requirements for deep learning training technology are getting higher and higher. In order to improve the efficiency of training, distributed deep learning has emerged. When training a distributed deep learning model, communication between computing nodes is required to realize the reduction processing of gradient information and ensure that the replica models of each computing node are updated synchronously. When the model is very large, the sample size is very large, and the scale of the computing cluster is very large, the communication time between computing nodes is long and the waiting time fo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/27G06F16/23
CPCG06F16/27G06F16/23
Inventor 张玉彦陈培张东
Owner LANGCHAO ELECTRONIC INFORMATION IND CO LTD